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Coordinated patterns of precisely timed action potentials (spikes) emerge in a variety 
of neural circuits but their dynamical origin is still not well understood. One hypothesis 
states that synchronous activity propagating through feed-forward chains of groups of 
neurons (synfire chains) may dynamically generate such spike patterns. Additionally, 
synfire chains offer the possibility to enable reliable signal transmission. So far, 
mostly densely connected chains, often with all-to-all connectivity between groups, 
have been theoretically and computationally studied. Yet, such prominent feed-forward 
structures have not been observed experimentally. Here we analytically and numerically 
investigate under which conditions diluted feed-forward chains may exhibit synchrony 
propagation. In addition to conventional linear input summation, we study the impact 
of non-linear, non-additive summation accounting for the effect of fast dendritic spikes. 
The non-linearities promote synchronous inputs to generate precisely timed spikes. We 
identify how non-additive coupling relaxes the conditions on connectivity such that it 
enables synchrony propagation at connectivities substantially lower than required for 
linearly coupled chains. Although the analytical treatment is based on a simple leaky 
integrate-and-fire neuron model, we show how to generalize our methods to biologically 
more detailed neuron models and verify our results by numerical simulations with, e.g., 
Hodgkin Huxley type neurons. 

Keywords: synchrony, networks, synfire chains, spil<e pattern, mathematical neuroscience, non-additive coupling, 
non-linear dendrites 



1. SPIKE PAHERNS AND SIGNAL TRANSMISSION IN 
NEURONAL CIRCUITS 

Reliable signal transmission is a core part of neuronal process- 
ing. A common hypothesis states that activity propagating along 
neuronal sub-populations that are connected in a feed-forward 
manner may support such signal transmission. Indeed, there is 
strong indication that activity propagation along feed-forward 
structures drives the generation of bird songs (Long et al, 2010) 
and experiments have shown propagation of synchronous and 
rate activity in feed-forward networks (FFNs) in vitro (Reyes, 
2003; Feinerman et al., 2005; Feinerman and Moses, 2006). 
Sequential replay in the hippocampus and in neocortical net- 
works also suggest underlying feed-forward mechanisms (August 
and Levy, 1999; Nadasdy et al, 1999; Lee and Wilson, 2002; 
Leibold and Kempter, 2006; Xu et al, 201 1; Eagleman and Dragoi, 
2012; Jahnke et al, 2012) and propagation of synchronous activity 
along feed-forward chains is a possible explanation for exper- 
imentally observed precise spike timing in the cortex (Riehle 
et al, 1997; Kilavik et al, 2009; Putrino et al, 2010). Further, 
the modular, hierarchical structure of many sensory and motor 
systems suggests propagation over sequences of areas in feed- 
forward manner, e.g., in bottom-up signal transfer (Felleman and 
Van Essen, 1991; Scannell et al, 1999; BuUmore and Sporns, 2009; 
Kumar et al, 2010). 

Feed-forward structures which support the propagation of 
synchronous activity are termed synfire chains. The concept 



was introduced by Abeles (1982) as groups of neurons (layers) 
with dense anatomical connections between subsequent groups 
that are embedded in otherwise roughly randomly connected 
local neural circuits. Two major questions regarding the dynam- 
ical options for synfire activity include a) how synchrony may 
actively propagate and b) how such spatio-temporally coordi- 
nated spike timing may be robust against irregular background 
activity, because the synfire chains are part of a cortical network 
with dynamics defined by the so-called irregular balanced state 
(van Vreeswijk and Sompolinsky, 1996, 1998). 

Addressing these points, theoretical studies have established 
conditions for stable propagation of synchrony in synfire chains 
(Diesmann et al, 1999; Gewaltig et al., 2001). Most synfire chain 
models assume functionally relevant FFNs that exhibit a very 
dense, often all-to-all connectivity between subsequent layers 
(Aviel et al., 2003; Mehring et al, 2003; Kumar et al, 2008) 
(see also a recent review on this topic Kumar et al., 2010). 
Such highly prominent feed-forward-structures, however, have 
not been found experimentally. Since cortical neural networks 
are overall sparse (e.g., Braitenberg and Schiiz, 1998; Holmgren 
et al, 2003), we may also expect some level of dilution for embed- 
ded feed-forward chains. So far, computational model studies 
assumed that such chains created from existing connections in 
sparse recurrent networks exhibit strong synaptic efficiencies 
and specifically modified neuron properties to enable synchrony 
propagation (Vogels and Abbott, 2005). 
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Recently, we have shown that non-additive dendritic interac- 
tions promote propagation of synchrony (Jahnke et al., 2012). 
The non-additive dendritic interactions considered are medi- 
ated by fast dendritic spikes (Ariav et al., 2003; Gasparini et al., 
2004; Polsky et al, 2004; Gasparini and Magee, 2006): upon 
stimulation within a time interval less than a few milliseconds, 
dendrites are capable of generating sodium spikes. These induce 
a strong, short and stereotypical depolarization in the soma. If 
this depolarization elicits a somatic spike, the spike occurs a 
fixed time interval after stimulation with sub-miUisecond preci- 
sion. This dendritic non-linearities relax the requirement of dense 
feed-forward anatomy and thereby allow for robust propagation 
of synchrony even in diluted FFNs with synapses of moderate 
strength within the biologically observed range. 

In the present article, we analytically and numerically inves- 
tigate in detail under which conditions synchronous activ- 
ity may reliably propagate along the layers of an FFN where 
the inter-group connectivity is diluted, as may be expected 
when they are part of a sparse cortical network. An embed- 
ding network is mimicked by external, noisy input. We study 
the influence of the network setup, including the influence 
of the emulated embedding network, and of different types 
of standard linearly additive as well as non-additive dendritic 
interactions. 

We derive analytical estimates for the critical connectivity — 
the minimal connectivity that allows robust propagation of syn- 
chrony. Some fundamental analytical results, in particular the 
ansatz for deriving a critical connectivity in the first place, have 
been briefly reported before (Jahnke et al., 2012). Here, we extend 
the approach and show how the bifurcation point, i.e., the transi- 
tion point from the non-propagating to the propagating regime, 
can be estimated quantitatively from the neurons' ground state 
properties. We investigate the validity range of the analytical 
predictions and check them via direct numerical simulations. 
Furthermore, we discuss the applicability of our results to bio- 
logically more detailed neuron models and network setups. In 
particular, we argue that the assumptions underlying the ana- 
lytical approach are met by a wide class of neuron models, 
including, e.g., conductance based leaky integrate-and-fire and 
Hodgkin-Huxley-type neurons. 

The article is structured as follows: After introducing the neu- 
ron model and network setup in section 2, we study in the main 
part the propagation of synchrony in linearly coupled FFNs (sec- 
tion 3.1) and in FFNs incorporating dendritic non-linearities 
(section 3.2). In particular, we derive tools to study the system 
analytically, compare the results to computer simulations and 
elaborate differences of the dynamics of FFNs with and without 
non-additive dendritic interactions. In the final part (section 3.3), 
we discuss the application of our analytical results to biologically 
more detailed neuron models. 

2. METHODS AND MODELS 
2.1. NEURON MODEL 
2.1.1. Linear model 

Consider networks of leaky integrate-and-fire neurons that inter- 
act by sending and receiving spikes via directed connections. The 
state of neuron k at time t is described by its membrane potential 



Vk(t) and its dynamics satisfy 



dVkit) 
dt 



Vk(t) 



(1) 



where is the membrane time constant of neuron k, 7™"^' := 
/°/t™ a constant input current, I^'^(t) the input current caused 
by spikes within the network and /^''*(f) the input current arising 
from spikes from external sources. When the neuron's mem- 
brane potential reaches or exceeds the threshold ©it its membrane 
potential is reset to V^'^*'^' and a spike is sent to the postsy- 
naptic neurons n, where it changes the postsynaptic potential 
after a delay x„k. After emitting a spike at t = to the neuron 
becomes refractory for a time period f"^*^^, i.e., Vj:(f) = V,''^'"^' for 

f € [fo, fo + f''^- 

To keep the model analytically tractable, we model the fast rise 
of the membrane potential upon the arrival of presynaptic spikes 
by instantaneous jumps of the membrane potential, such that the 
resulting input current reads 



Im 



(2) 



Here ejt/ denotes the coupling strength from neuron I to neuron k, 
is the mth spike time of neuron I and x^i specifies the synap- 
tic delay. In addition to spikes from the network each neuron 
receives excitatory and inhibitory random inputs that emulate an 
embedding network. These external inputs are modeled as ran- 
dom Poisson spike trains with rate v™*^ and v'"*^, respectively. The 
resulting input current is given by 



km 



(^t 



exty inh \ 
km } 



(3) 



where f^^^''^"'^ (^tm''"'^) is the arrival time of the mth excitatory 



km ^ km 

(inhibitory) spike to neuron k and e*^ 
corresponding coupling strength. 



> 0 (c'"'' < 0) denote the 



2.1.2. Non-linear model 

In the above model all input currents are summed up linearly. 
To also investigate the effect of dendritic spikes we modulate the 
sum of synchronously arriving excitatory inputs by a non-linear 
dendritic modulation function aj^i (•). This can be directly read 
off from experimental data (Ariav et al., 2003; Gasparini et al, 
2004; Polsky et al, 2004; Gasparini and Magee, 2006): If the sum 
of excitatory inputs is below the dendritic threshold @h, the single 
inputs are processed linearly (oni (■) equals the identity). If the 
sum of inputs exceeds the dendritic threshold ©;,, the depolariza- 
tion is strongly non-linearly enhanced compared to that expected 
from linear summation. This is, in biological terms, due to a 
dendritic spike elicited. Larger inputs have been experimentally 
found to not further increase the somatic peak depolarization. 
The dendritic modulation function may then be modeled as 



6 for e < @h 
K otherwise 



(4) 
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The dendrites process synchronous inputs non-additively: inputs 
below the dendritic threshold are summed linear, inputs above 
this threshold are summed supra-linear and, due to the satura- 
tion, very large inputs are summed sub-linear. 

If not stated otherwise, we consider only exactly simultaneous 
arriving spikes as sufficiently synchronous; to allow for exactly 
simultaneous arrivals, the synaptic delays are chosen as homo- 
geneous tH = X. The input currents caused by spikes that are 
received from the network are then given by 



Then 



/€Mexc(tf) 



EH 



ieMi„i(tf) 



&(t-/-i). (5) 



Here, the sum over / denotes the sum over all times at which 
spike(s) are sent in the network, irrespective of which neuron(s) 
is (are) spiking. The sets Mgxc {ff^ and M^^h (t^) specify the set 
of neurons that send an excitatory or inhibitory spike at time /, 
respectively. (To describe a network with linear dendrites (Tni(6) 
is replaced bye). 

In section 3.3.1 we consider inhomogeneous delay distribu- 
tions and finite dendritic integration window Af (i.e., non-linear 
amplification of inputs received within finite time interval Af) 
and discuss how the results achieved for homogeneous systems 
can be generalized. 

2.2. NETWORK TOPOLOGY 

We consider the propagation of synchrony in diluted Feed- 
Forward-Networks (FFNs, synfire-chains). They consist of a 
sequence of m layers, each composed of m neurons. Neurons 
of one layer form excitatory projections to the neurons of the 
subsequent layer with probability p; the strength of an existing 
connection from neuron I to neuron k is denoted by ejt;. 

For simplicity of presentation, we consider homogeneous 
neuronal populations, i.e., all neurons have identical properties 
(t^ = z'^,@k = ® and = V'""^ for all i), as well as homo- 
geneous coupling strengths, i.e., = € if a connection is realized, 
throughout this article. If not stated otherwise, we use t™ = 
14 ms and 0 = 15 mV as standard values for the membrane time 
constant and the neuron threshold. 



2.3. GROUND STATE DYNAMICS 

We consider networks, where the single neurons are placed in a 
"fluctuation driven regime," i.e., in the ground state the average 
input to each neuron is sub-threshold and spiking of neurons is 
caused by fluctuations of the inputs. This setup allows to emulate 
the dynamics of neurons which are part of a balanced network 
(van Vreeswijk and Sompolinsky, 1996, 1998). The neurons fire 
asynchronously and irregularly with low firing rate v; the spike 
trains resemble Poissonian spike trains (Tuckwell, 1988; Brunei 
and Hakim, 1999; Brunei, 2000; Burkitt, 2006). Thus, the inputs 
to the neurons may be described by three Poissonian spike trains 
with rates (external, excitatory), v'"^ (external, inhibitory) 
and v'°' = vpti) (inputs from the preceding layer). Since the num- 
ber of inputs Nj, X e {exc, inh, int}, in a time interval T is 
Poisson distributed, the expected number of inputs (Nj) and the 



variance {(N: 



x 



{N§)f), equal v^r : 



(1 = 7o + t" 



i.yinh^inh _j_ 



(6) 



is the mean of the total input to the neurons in an interval of the 
size of the membrane time constant, T = t™, and 



,inh 



-I- T^pMve^ 



(7) 



is its variance. In diffusion approximation, the distribution of 
membrane potentials Pv{V) and the mean firing rate v can 
be derived analytically (Brunei and Hakim, 1999; Brunei, 2000; 
Helias et al, 2010). In particular, for networks with low firing 
rates the probability density of membrane potentials (see, e.g., 
Tuckwell, 1988) 



PviV) 



1 



: exp 



V-[i 



(8) 



is Gaussian and can be expressed in terms of the input current. In 
this approximation the average firing rate is 



1 0 



exp 



&-[L 



(9) 



and depends on |x and a only via the quotient 



0-11 



(10) 



which is the distance of the average input /x from the neurons' 
threshold 0 normalized by the standard deviation 0 of the input. 
For the analytical derivations throughout this article we focus on 
the regime of low spiking rates (a ^ 2; v ^ 1.5Hz). 

In the absence of synchronous activity each neuron receives a 
large number of inputs from the external network and only a few 
inputs from the previous layer of the FFN, such that the ground 
state dynamics of the network is mainly established by the exter- 
nal inputs. To keep the input balanced we choose v*^" 



,,inh 



and 6*^ 



cinh . 



throughout the article. 



2.4. PROPAGATION OF SYNCHRONY 

To initiate propagating synchronous activity along the considered 
diluted FFN, we excite in the first layer a subgroup of ^0 < w neu- 
rons to spike synchronously. This causes a synchronous input to 
the following layer after the synaptic delay x and may therefore 
initiate synchronous spiking of a subgroup of neurons in that 
layer. These may again excite synchronous spiking in the next 
layer and so on. Depending on the ground state, i.e., the layout 
of the external network, on the layer size w, and on the coupling 
strength €, a synchronous pulse may or may not propagate along 
the FFN (cf Figures 1A,B,D,E). 

In addition to the triggered propagation, one might generally 
also expect the occurrence of spontaneous propagation of syn- 
chronous activity: Neurons of a particular layer share inputs from 
the previous layer and this causes correlations in their spiking 
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FIGURE 1 I Propagation of synchrony in diluted FFNs. (A,B,D,E) Raster 
plots of diluted feed-forward networks (m= 10, a) = 200, e = 0.25mV). 
Witfi increasing connection probability p propagation of synchrony can be 
enabled (A,B) in networks with additive (linear) and (D,E) in networks with 
non-additive (non-linear) dendritic interactions (0f, = 4mV, k= 11 mV). (C,F) 
Average number of synchronously active neurons in the second layer, 



92, vs. the number of synchronously active neurons in the initial layer, gi; 
panel (C) linear, panel (F) non-additive dendritic interactions (average over 
n= 10,000 trials: solid line, transition probability: shading). Note that 
non-linear dendrites allow for sparser connectivity, (E) vs. (B) and for a 
sparser code, i.e., for smaller numbers of spiking neurons in an activated 
group, (F) vs. (C). 



activity. Over the layers these correlations can accumulate and 
lead to synchronous spiking (Aviel et al., 2003; Rosenbaum et al, 
2010, 2011; Litvak et al., 2013). However, in the setups consid- 
ered in this article, the effect is negligible due to two reasons: (1) 
each neuron receives a large number of external (uncorrected) 
inputs and this background noise has a decorrelating effect, (2) 
we study the system near the critical point, i.e., for parameters 
where even synchronized spiking of all neurons of a particu- 
lar layer is just sufficient to initiate a propagation of synchrony. 
Thus, spontaneous propagation of synchrony effectively does not 
occur. 

We study the transition from the non-propagating to the 
propagating regime by means of a iterated map that yields the 
expectation value of the number of synchronously spiking neu- 
rons gi+\ in layer ; -|- 1 if gi neurons are synchronously active 
in layer i. There is always one trivial fixed point. Go, of this iter- 
ated map with 0 = Go = + 1 = , which corresponds to absent 
activity. If gi+i < gi for all gi > Go, synchronous activity will die 
out after a small number of layers. If ^, -|- 1 > gi for some substan- 
tial group size, gi > Go, a stable propagation of synchrony may 
be enabled (cf Figures 1C,F). More precisely, we will show in 
this article that with increasing connectivity p the system under- 
goes a tangent bifurcation and two fixed points Gi and G2 > Gi 
appear. If existing, Gi is always unstable (the diagonal is crossed 
from below; the slope of the iterated map needs to be larger than 
one) and G2 is always stable [all connections within the FFN are 
excitatory such that the iterated map is monotonicaUy increasing 
(slope larger than zero, in particular larger than —1)]; further at 
G2 there is an intersection with the diagonal from above thus the 
slope is smaller than one and stationary propagation with group 
sizes around G2 is enabled. 



In computer simulations, we determine for each given net- 
work setup by the following procedure whether a propagation 
is possible: after some initial time f""' we excite all neurons of 
the first layer to spike synchronously and measure the num- 
ber of active neurons gi in the ith layer at the expected spiking 
time f™'' = f""' -|- ix. If gi is substantially larger than the num- 
ber of active neurons arising from spontaneous activity in more 
than 50% of n trials (i.e., n repetition of the same simulation 
with different initial conditions), we denote the propagation of 
synchrony as successful. The critical connectivity p*, that marks 
the transition from a regime where propagation of synchrony 
is not possible to a regime where propagation of synchrony is 
enabled, is found by determining the lowest connection prob- 
ability p for which an initial synchronous pulse propagates 
successfully. 

As the connections within the FFN are all excitatory, it is suffi- 
cient to check whether propagation of synchrony can be initiated 
by inducing synchronized spiking of all eo neurons of the first 
layer: Stationary propagation of synchrony can be enabled if there 
is a non-trivial stable fixed point (G2) of the iterated map for the 
average group size. For purely excitatory connections the basin of 
attraction of this fixed point is bounded from the left by an unsta- 
ble fixed point (Gi) and from the right by the maximum group 
size given by the layer size w. 

3. RESULTS AND DISCUSSION 

Under which conditions can synchronous signals propagate 
robustly along diluted FFNs? To answer this question in detail, 
we first focus on networks with linear dendrites. Afterwards we 
study the propagation of synchrony in networks incorporating 
non-additive dendritic interactions and compare with the linear 
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case. Finally, we show that the derived results are directly appli- 
cable in biologically more detailed neuron models and network 
configurations. 

3.1. FFNs WITH LINEAR DENDRITES 

In this section, we consider linearly coupled FFNs. In the first 
part, we derive analytical estimates for the critical connectivity 
P2 that marks the transition from the non-propagating to the 
propagating regime; the initial steps follow the lines of lahnke 
et al. (2012); Memmesheimer and Timme (2012). In the second 
part we investigate the influence of the external network on the 
propagation of synchrony and determine the parameter-region 
for which the analytical estimates are applicable. In particu- 
lar, we show that the derived estimates are applicable in the 
biologically relevant parameter-region, where the spontaneous 
firing rate is low and the distribution of membrane poten- 
tials is sufficiently broad. Finally, we study how the properties 
of propagating synchronous pulses depend on different system 
parameters. 

3. 1. 1. Analytical derivation of critical connectivity 

To access the properties of propagation of synchrony we consider 
average numbers of active neurons in the different layers of an 
FFN: for this, we derive a iterated map which yields the expected 
number of neurons that will spike synchronously in one layer 
given that in the preceding layer a certain number of neurons was 
synchronously active. 

If in the I'th layer, gi neurons spike synchronously, the num- 
ber of synchronous inputs h a single neuron in layer i + 1 receives 
follows a binomial distribution h ^ B (g;, p). We denote the spik- 
ing probability of a single neuron due to an input of strength x 
hjpf (x). The average or expected spiking probability p^P (g,) of a 
single neuron in layer z -|- 1 is then given by 

/p (g,) = E[pf m\g,] = f2(t)p' i^-pf~'pf -(11) 

h = 0 ^ ^ 

Here and in the following we denote the expectation value 
of a function /(X) of a random variable X by E[/(X)]; 
conditional expectations are denoted by E[/(X)|y]. The 
expected number of spiking neurons in layer z 4- 1 is then 
simply 

E[ft+ik,] = «/Pfe) (12) 

= ^Y.[''i)p\'-pr'vf^^^)- (13) 

f! = 0 ^ ^ 

If the connection probability p is low and/or the connection 
strengths « are small, the spontaneous spiking activity in the 
absence of synchrony is only weakly influenced by the spiking 
activity within the FFN. Thus as a starting point, we assume 
that the ground state is exclusively governed by external inputs 
(effectively setting 6y = 0). Then, the mean input to the neurons 
in an interval of length t™ is (jL = 7o with standard deviation 
cr = 6™* V 2t'"v™' (cf section 2.3). Using the probability den- 
sity (Equation 8), we calculate the spiking probability of a single 



neuron, pj (x), due to an input of strength x\ 



Pv (V) dV 



Erf 



"0 - |X" 


-Erf 


& — [)u+ X 


) 


a 




a 





(14) 
(15) 



equals the probability of finding a neuron's membrane potential 
in the interval [0 — x, 0]. To derive a iterated map for the aver- 
age number of active neurons (which maps E\gi\ — > -E[g,+ i]), 
we interpolate E 1 [ g,] for continuous and in the second 
step replace g, by its expectation value E [^,] . The fixed points, 
E 1 1 E \giW = E [^,] , qualitatively determine the propagation 
properties of synchronous activity. In the rest of the manuscript 
we are dealing with the average number of active neurons in a 
given layer. Therefore, for simplicity we denote the expectation 
value of the average number of active neurons in a given layer i by 
gi instead of E [^,] . 

For sufficiently small connection probabilities p the map 
(Equation 12) has only one (trivial) fixed point Go = gi+ i = 
g^ = 0. Any initial synchronous pulse will die out after a small 
number of layers (see also Figure 1). With increasing connectiv- 
ity two additional fixed points Gi (unstable) and G2 > Gi (stable) 
appear via a tangent bifurcation. 

For FFNs with purely excitatory couplings between the layers, 
the second fixed point G2 (if it exists) is always stable: The spik- 
ing probability pf (x) is monotonically increasing with input x and 
thus also the iterated map (Equation 13) is monotonically increas- 
ing (i.e., the slope is larger than 0). Moreover, if G2 exists the slope 
of the iterated map at this intersection point with the diagonal 
is smaller than 1. This implies that G2 is stable and synchronous 
pulses of size gi > Gi typically initiate a propagation of synchrony 
with an average number of active neurons around G2. The criti- 
cal connectivity p^ at the bifurcation point marks the minimal 
connectivity that allows for stable propagation of synchrony. 

Although the distribution of inputs from one layer to the 
subsequent one and the spiking probability of a single neuron 
pf(-) are known, there is no analytic closed form solution to 
the fixed point equation gi^i = gi = g*. In other words, we can 
compute the firing probability pf (xq) for any xq, and therefore 
also E [gi+ 1 1 g,] for anyg, hutg* = E [gi+ 1 1 g*] is transcenden- 
tal. We thus derive an approximate solution. We choose some 
expansion point gi (see section 3.1.2 for details), and approxi- 
mate the function E [gi+ 1 1 g*~\ by a polynomial gi + S(g* — gi) 
in second order in (g* — gi) near gi. The arising quadratic fixed 
point equation g* = gi + S(g* — gi) is then analytically solvable 
in g* . This also allows to analytically compute the critical con- 
nectivity p*^: it is the parameter value at which the iterated map 
undergoes a tangent bifurcation, i.e., at which the two solutions 
of the fixed point equation become equal upon changing from 
complex-conjugate to real. Since the right hand side of Equation 
(13) does not offer itself for a direct series expansion in g* , we 
derive gi + S(g* — gi) from an appropriate expansion of pf (he) 
and a subsequent computation the arising expectation values. 

In biologically relevant scenarios, the neurons usually receive 
a large number of synaptic inputs and thus the distribution of 
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membrane potentials -Py(V) is broad, PviV) changes slowly with 
V. Then, Pv{V) around some V = Vq can be approximated by 
considering a series expansion with a small order and it is pos- 
sible to derive an approximation for the critical connectivity 
based on an expansion of pf{-). Expanding pf{x) into a Taylor 
series around some xq and using Equation (12) yields 



gi+i = mE 



T^—(hi-xor 

^ ' Ml 



(16) 



^ Pf'^ (xo) 

^^E[(/t€-Xo)"U,]- (17) 



Here and in the following we denote the «th derivative of a 
function /(x) atx = xq by 



d". 



-fix) 



(18) 



lx = xo 



Replacing the derivatives of pf(-) by the (one order lower) 
derivatives of probability density of membrane potentials Pv(V) 
according to Equation (14) yields 



i = uipf (xq) + a 



V 



(Vo) 



(-l)"-i«! 

n= 1 ^ ' 



where we defined 



E[(/i€-xo)"U,],(19) 



(20) 



for better readability. 

We have recently shown (Jahnke et al, 2012) that it is possible 
to derive a scaling law for the critical connectivity using 



(21) 



the (unknown) average input from one layer to the next during 
stationary synchrony propagation, as expansion point. For this 
choice the expectation value E [ (/le — xq)" [ gi\ in Equation (19) 
simplifies to 

E [(/!€- xo)"| =€"E[(;i-E[?i])"l^,] =€"m„, (22) 

where we denote by m„ the nth central moment of the Binomial 
distribution B (g,, p), specifying the distribution of inputs to the 
(z -|- l)th layer. In the limit of large layer sizes eo and small cou- 
pling strengths € keeping the maximal input tw to each layer 
constant (to preserve the network state), all summands for n > 2 
vanish, and Equation (19) simplifies to 



g,+ i= (^pf{g,pi) ■ 



(23) 



Using the implicit function theorem one can show that this 
implies the scaling law 



Pi 



1 

>.€C0 



(24) 



where X is a constant independent of 6 and m (Jahnke et al., 2012). 
We note that for the derivation of the scaling law (Equation 24) we 
did not use the actual functional form of the distribution of mem- 
brane potentials Pv(V). Therefore this estimate holds ifPv(V) is 
sufficiently slow changing with V such that the Taylor expansion 
(cf Equation 16) is applicable, but its validity is not restricted to 
the low-rate approximation. 

However, the dependence of the prefactor 1 /X on the layout 
of the external network remained unknown. Here, we present an 
approach that enables us to derive an approximate value for X. 
We consider the expansion (Equation 19) around xq up to second 
order, 

gi+ 1 ^ mpfixo) + mPv (Vo) • {tg,p - Xq) 



[{ig.p-xof + t^g,p{l-p)] (25) 



The truncated series (Equation 25) is quadratic in^, such that the 
fixed points g*^2 = gi+i = gt can be obtained analytically. 



* 1 
8i.2 = Yi± 



vi- 



xo (iPviVo) + xoP'^^ (Vo)) - 2pf(xo) 



(26) 



where we defined 



(2 (Pv(Vo) + xoP';)\Vo)) + (p - 1) P^v^iVo)^) 



(27) 



At the bifurcation point, the root in Equation (26) vanishes 
such that both fixed points agree (g* = g| ) and yi = = g| 
specifies the average size of a propagating synchronous pulse. 
Consequently, the critical connectivity is obtained by choosing p 
such that 



y1 



which yields 



Xq 



(iPviVo) + xoP^'' (Vo)) - 2pf(xo) 



(28) 



Pl 



1 1 

2 ~ c 



X* 

p'^v^m 



(ei'v'(Vo)-2X*) 
4(P<;\V0))' 



(29) 



where we defined 



Pv{Vo)+xoP2\Vo) 



(30) 



-^p'i)\Vo) (xo (iPviVo) + xoPj." (Vo)) - 2pf(xo)) 

which is independent of the setup of the FEN and completely 
determined by the layout of the external network and the choice 
of the expansion point xq. 
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As before we consider the limit of large layer sizes to and 
small coupling strengths €, i.e., we replace m 
sider the leading terms of a series expansion of Equation (29). 
The expansion of the square bracket in Equation (29) yields 



pW(Vo) const 



+ 



X* • const 



l)+0{^% (31) 



such that the critical connectivity assumes the functional form 
given by Equation (24), 



Pi 



1 



X*6(jO 



(32) 



Thus X = X* defined by Equation (30) provides an approxima- 
tion of the constant X fully specifying the critical connectivity p^- 

3.1.2. Optimal expansion point 

To derive Equation (30) we assumed that it is sufficient to con- 
sider the second order expansion of pf{x). It is thus necessary 
to choose an appropriate expansion point that results in fast 
convergence. In particular for the choice xq = x^, that we will 
now derive, Equation (37) below, the bifurcation diagram near 
the bifurcation point is well approximated already for /c = 2 (cf 
Figure 2). 

The size of a propagating group at the critical connectivity is 
Yi (cf Equation 27) and thus the resulting average input is pIyl^- 
Our expansion point xq should lie near to this value, which is, of 
course, unknown prior to solving the fixed point equation. We 
will thus compute a range in which p^Yi* t° choose 
the expansion point appropriately within. We assume that m is 
large and employ Equation (23) which allows an direct estimate 



of this range as we know the functional form explicitly. Equation 
(23) with g,_f_i = gi is just another transcendental equation for 
the fixed points and it has zero, one, or two non-trivial fixed 
point solutions points g* and g*, which are then also solutions 
of Equation (19) with gi+i = gi- At the bifurcation point (g* = 
g* = ^|) where the diagonal is touched, the function pfigpt) has 
to be concave and monotonic increasing with respect to g. The 
definition (Equation 14) of pf(x) implies that it is monotonic 
increasing for all x > 0. Moreover, it is concave for all x > 6 — [jL, 



pj.^' (x) = Pv(& - x) > 0 for X > 
pj^' (x) = (0-x) <0 for X > © - (X, (34) 



0 



(33) 



such that the bifurcation point satisfies 



Xo > © — 



(35) 



The condition Equation (33) holds because -Py(V) > 0 is a prob- 
ability density and Equation (34) is derived directly from differen- 
tiating Equation (8). To maximize the quality of the second order 
approximation Equation (25), we choose xq = xj such that the 
contribution to the expansion (Equation 19) of the k = 3rd order 
term equals zero. According to Equation (19), all 3rd order terms 

(2) 

are proportional to Py (© — xq); so we determine the expansion 
point Xg as a deflection point of Pv(-), requiring that the second 
derivative of Pv(V) vanishes for V = © — xJ , 



(3) . d^PviV) 



(36) 



In the considered regime of low spiking rates, we find Xg = © 
(JL ± cf Equation (8). Due to Equation (35) 



©-|i,-|- 



(37) 
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FIGURE 2 I Iterated map and bifurcation diagram for the average group 
size of a propagating syncfironous pulse. (A) Iterated map (Equation 19) 
truncated after expansion order k (color code) witli xo = (cf. Equation 
37). (B) Fixed points of the iterated nnaps shown in (A); with increasing 
connectivity two fixed points appear by a saddle node bifurcation. We note 
that already a second order expansion (red), i.e., the lowest order at which a 
saddle node bifurcation can occur, approximates the bifurcation diagram 
(blue) near the bifurcation point well. 



For Xo = Xg the bifurcation diagram near the bifurcation point is 
well approximated already for /c = 2 (cf Figure 2) and Equation 
(30) provides a good estimate of the critical connectivity p^ (cf 
Figure 3). 

3. 1.3. Influence of external network 

In the previous section we derived an iterated map for the average 
group size (cf Equation 13) and an approximation for the critical 
connectivity p2 (cf Equations 30 and 32) that marks the transi- 
tion from FFNs which do not support propagation of synchrony 
to FFNs that do. In this section we focus on the robustness of our 
results. How does the critical connectivity change with the layout 
of the external network? For which parameter range does the esti- 
mate of the critical connectivity (given by Equations 30 and 32) 
yield reasonable results? 

The derivation was based on the assumption that the ground 
state dynamics of the neurons of the FFN is completely deter- 
mined by the external inputs. This assumption holds if the spon- 
taneous firing rate v of the neurons and/or the coupling strengths 
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FIGURE 3 I Critical connectivity in FFNs with linear dendrites decays 
algebraically with coupling strength t and layer size u. Tine parameters 
of the external inputs (emulated embedding network) are fixed (fo = 5 mV, 
v"''' = 3 kHz, t"^ = 0.5 mV). Panel (A) shows the critical connectivity p* vs. 
the layer size m for different coupling strengths (ce = {0.05 mV (red), 0.1 mV 
(cyan), 0.125 mV (green), 0.2 mV (blue), and 0.4 mV (black))) and panel (B) 
shows p*i^ vs. the coupling strength 6 for different layer sizes (w = {50 (red), 
100 (cyan), 150 (green), 200 (blue), and 400 (black)}). In the main panels we 
use a logarithmic scale, the insets have a linear scale. The squares indicate 
the connectivity above which a synchronous pulse propagates from the 1st 
to the 20th layer of a FFN in at least 50% of n = 30 trials. The critical 
connectivity given by Equation (32) (solid lines) with xo = Xg (cf. Equation 
37) is in good agreement with computer simulations. As predicted 
Pi oc (e(o)"^ and the proportionality factor 1 /X is well approximated by the 
estimate 1/1* derived in Equation (30). 



6 and/or the connectivity p are sufficiently small. We will gener- 
alize our approach and show how the impact of preceding layers 
on a layer's ground state can be taken into account. Thereafter we 
wUl compare the results with computer simulations, identify the 
regions in parameter space for which the derived approximations 
hold and discuss deviations between direct numerical simulations 
and analytics. 

The first layer of an FFN receives inputs only from the exter- 
nal network and according to Equations (6, 7) the mean [Li and 
standard deviation 01 of its input is 



1^1 = h 



(38) 
(39) 



as assumed in the previous section. All following layers receive 
external inputs and spikes from their preceding layer(s). The 
mean |x„ and standard deviation a„ of the input to neurons of 
the nth layer (with n > 2) reads (cf Equations 6 and 7) 



(40) 
(41) 



Here we denote the spontaneous firing rate (in the absence of syn- 
chrony) of neurons of the (n — l)th layer by v„_ 1. It is given by 
Equation (9) as 



1 6) - >JL„- 

TTt™ cr„_i 



exp 



Q - V-n- 



(42) 



From layer to layer, the mean input, the standard deviation as 
well as the firing rate increase. For setups, where the ground state 
of the FFN is non-pathological, i.e., the firing rates of all lay- 
ers are bounded, the additional corrections AX„ := X„ — X„_ 1 
for X e {[I, a, v} decrease with n, and [in, a„ and v„ saturate for 
sufficiently large n. Thus, (jLoo and 0^0 describe the input to the 
neurons of an infinitely long FFN and the single neurons of such 
an FFN spike with an average rate v^o- Accordingly, replacing |i 
and a by jXoo and Ooo in Equation (13) [where they appear as 
parameters oipf(-)\ yields an iterated map for the average group 
size. 

In Figure 4, we compare the critical connectivity found by 
numerically determining the bifurcation point of the iterated 
map (Equation 13) (i.e., we determined the connectivity p for 
which the iterated map touches the diagonal; solid lines) with 
computer simulations of propagating synchrony (markers). To 
also cover scenarios, where the input from the preceding layer 
is not negligible, we consider infinitely long FFNs (then, the 
distribution of membrane potentials is equal in all layers). In 
computer simulations this can be approximated by a sufficiently 
long FFN with periodic boundary conditions, i.e., an FFN where 
the last layer connects to the first layer. For moderate external 
inputs, i.e., moderate /q and €™', already the analytical results 
neglecting the influence of the preceding layers (using |Xi and 
oi) agree well with computer simulations (cf Figure 4A, solid 
lines). However, for large external inputs, i.e., large h and t^^^, 
the critical connectivity is overestimated. Here, the assumption 
that the distribution of membrane potentials is not influenced 
by the connectivity of the FFN does not hold. The additional 
input shifts the membrane potentials to higher values and con- 
sequently a lower connectivity is required for a propagation of a 
synchronous pulse. The corrections given by Equations (38-42) 
account for these deviations to some extent (cf Figures 4B,C; 
solid lines), in particular for setups where the spontaneous firing 
rate is low. However, for very large Iq and e'^''*, the critical con- 
nectivity is under-estimated. Here, the spontaneous firing rate is 
too high and the low-rate approximation. Equations (8-9), is not 
adequate to describe the system; the firing rate and thus the mean 
input from the previous layer are over-estimated. This becomes 
particularly clear in Figure 4C, where we show the critical con- 
nectivity as a function of the strength of the external inputs e'^'^'. 
For any given /q (different colors), the critical connectivity for 
small is well approximated; with increasing 6*^^ the firing 
rate increases [a decreases and thus v increases; cf Equations 
(9 and 10)] and when the coupling strengths exceed a 
/o -dependent threshold, the low- rate approximation becomes 
inapplicable. 

Applying the methods in Brunei and Hakim (1999); Brunei 
(2000), the firing rate and the distribution of membrane poten- 
tials can be derived in diffusion approximation for states with 
higher spontaneous firing rates. Although most of the analyti- 
cal considerations above are also applicable within this approx- 
imation, the determination of an optimal expansion point 
(cf. Equations 36 and 37) becomes more difficult and a closed 
form expression does not exist. However, the critical connectiv- 
ity can be obtained by numerically determining the fixed points 
of the iterated map (Equation 13) and we find that it agrees with 
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FIGURE 4 I Robustness of analytical estimates of the critical 
connectivity. (A-C) We consider tine critical connectivity p* of infinitely long 
FFNs, that are approximated by an FFN (m = 20, m = 150, e = 0.2 mV) with 
periodic boundary conditions in direct numerical simulations (markers), for 
different layouts of the external network. Panels (A,B) show pj vs. b for fixed 
c""' and panel (C) shows pj^ vs. e*"" for fixed fo. The solid (colored) lines 
indicate the critical connectivity found by numerically determining the 
bifurcation point of the iterated map (Equation 13). In panel (A) we neglect 
the influence of previous layers on the ground state of a considered layer in 
the analytical computations [i.e., we use (ii and oi, of. Equations (38) and 
(39)1. In (B,C) we employ corrections to account for their influence, of 
Equations (38-42). We show the third order correction, higher orders add 



only small modifications to the curves, but the numerical computations get 
more costly. The thick gray lines in (B,C) indicate the bifurcation point of the 
iterated map (Equation 13) with Pv( V) derived from the diffusion 
approximation of leaky integrate-and-fire neuron dynamics with Poissonian 
input (Brunei and Hakim, 1999; Brunei, 2000). The dashed lines are the 
estimates of the critical connectivity given by Equations (30 and 32). Again, in 
panel (A) we neglect the influence of previous groups on the ground state, in 
panels (B,C) we use the third order correction. The estimates agree with the 
data from numerical simulations within the biologically relevant parameter 
range, where (1) the spontaneous spiking activity is low and (2) the 
distribution of membrane potentials is sufficiently broad. For further 
explanations see text (section 3.1.3). 



computer simulations for the entire considered range of 7o and 
€'^'*, (c£ Figures 4B,C; gray lines). 

Analogous to the approach presented above, corrections for 
the influence of preceding layers can be taken into account for 
the analytical estimate of the critical connectivity derived in the 
previous section (Equations 30 and 32). Replacing the connectiv- 
ityp by the approximation pj^ = (X*tm)^^ in Equations (40, 41) 
yields 

li.„ = 7o + t:™/X*_iV„_i (43) 
a„ = Jlv'^^^x'^ (€«t)2 + €V„_ it™/X*_ 1 (44) 

where )^*_ j := (|Xn- i, cr„_ i) is given by Equation (30) and 
v„_ 1 = V {[In- 1, Cn- i) is given by Equation (42). In Figure4 
we show the estimate of the critical connectivity = (^n^*^) ' 
(cf. Equation 32) using X* (panel a; dashed line), i.e., neglect- 
ing the influence of the preceding layers, and using a higher 
correction order (panel b,c; dashed line: third order). For suffi- 
ciently large e*^"' the critical connectivity found by numerically 
determining the bifurcation point agrees with the analytical esti- 
mate given by Equation (32). As discussed above, the correc- 
tions Equations (43, 44) account for the deviations from the 
simulated data as long as the total spontaneous firing rate is 
sufficiently low. However, for small t™' the critical connectivity 
is under-estimated. Here, the standard deviation of the inputs 
(cf Equation 7) is low, such that the distribution of membrane 
potentials Pv(V) is narrow [for 6™' 0: Pv(V) ^ S (V - |x); 
cf. Equation (8)], the spiking probability of one neuron, pf(-), 
increases steeply in a small interval [for 0: pf(x) 

0 (x — |i); c£ Equation (8)] and thus the approximation ofp^(-) 



by the leading terms of a Taylor expansion is not sufficiently 
accurate. 

However, in the biologically plausible parameter regime, where 
the firing rates are small and the distribution of membrane 
potentials is broad, the critical connectivity is approximated 
well by Equation (32) together with Equation (30) (defin- 
ing X*), Equation (37) (defining Xq) and the corrections that 
account for the influence of the preceding layers. Equations 
(43, 44). 

3. 1.4. Characteristics of propagating synchronous pulses 

In the previous sections, we have shown that a synchronous pulse 
may propagate along a diluted FFN. In this section we study the 
characteristics and properties of a propagating synchronous sig- 
nal. We consider them at the transition to stable propagation, 
P1, because there they depend only weakly on the network setup. 
How large is the fraction of neurons that participate in propa- 
gating synchrony? How does this fraction depend on the network 
setup? 

To answer such questions, we consider the effect of a prop- 
agation synchronous pulse on the single layers in the network, 
as a measure for the effective pulse size. In other words, we con- 
sider the mean input [ii a neuron receives from the preceding 
layer if a synchronous pulse propagates along the FFN at the crit- 
ical connectivity It is given by the product of the connection 
probability pf, the connection strength 6 and the average size of 
a propagating synchronous signal yi; using Equations (27) and 
(29) yields 

„ Pv(e-x;) + p^»(e-x*)x*--K* 

= YLpie = 77-. (45) 

P\}\&-x*} 
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and after inserting X* as given by Equation (30), 



4 (2Pv(& ■ 



-4) + <Pv (0 



4))-2p/K*) 



p\)>i&-4) 



(46) 



According to Equation (46) the average input [ii to the neurons 
due to a propagation of a synchronous pulse is independent of 
the layer size co and coupling strength €. For setups with mod- 
erate external inputs (i.e., inputs of the preceding layer influence 
the neurons' ground state only weakly; see also section 3.1.3) the 
distribution of membrane potentials Pv(-) (c£ Equation 8), the 
firing probability of single neurons (c£ Equation 14) as well 
as the expansion point (deflection point of Pyi-); cf. Equation 37) 



@-Io + t'^'Wx'^v'^'^ 



(47) 



are fuUy determined by the external inputs (/q, v'^''' and €™'). 
Figures 5A,B illustrates the dependence of [ii on the layout of 
the external network and the FFN: as expected from our analyti- 
cal considerations, the dependence on the layer size and coupling 
strength is weak when Jq and are kept fixed. With increas- 
ing mean of the external input (/q) the distribution of membrane 
potentials Pv{V) is shifted toward the threshold 0, such that 
it is more likely to find the membrane potential of the neurons 
near the threshold and the critical connectivity decreases (cf. also 
Figures 4A,B). Naturally this implies a decreasing average input 
\iL atp|, which is shown in Figure 5A for different external cou- 
plings i'^^ and parameters of the FFN. Increasing the external 



coupling strength 6™* (and with it the variance of the exter- 
nal input) causes a broadening of the distribution of membrane 
potentials; the membrane potentials of some neurons are shifted 
toward the threshold and the membrane potentials of other neu- 
rons are shifted away from it. If the fraction of neurons that 
participate in the propagation of the synchronous pulse is large, 
this implies an increasing critical connectivity (Figure 5B; cf. also 
Figure 4C). 

The spiking probability of a single neuron due to the mean 
input |Xi equals the average fraction p^'^'^ of neurons of one layer 
that participate in a propagating synchronous pulse, 



^frac 



■■ Pf (M-l) ■ 



(48) 



Interestingly, in the considered regime of low spiking rates and 
sufficiently broad distribution of membrane potentials, where 
the approximations given in section 3.1.1 are applicable, p*^^'^ 
depends on the setup of the external inputs only via the quotient 
a = " ~ ^ (cf Equation 10), or, equivalently, on the spontaneous 
firing rate v of the neurons (cf Equation 9). This can be shown by 
combining Equations (8, 37) and (Equation 46), 



en \ 1/4 



-Erf(-L)-Erf(.) 



1/2 



(49) 
(50) 




FIGURE 5 I Properties of propagating synclironous pulses at the 
transition from the no-propagation to the propagation regime. Panels 
(A,B) show the mean input \^l that a layer receives due to a propagating 
synchronous pulse in the preceding layer. \t.i measures the effective pulse 
size (the impact of a propagating synchronous pulse) and is mainly determined 
by the external inputs rather than by the setup of the FFN. In (A) the variance 
of the external input (measured by e^"*') is fixed and \>.i^ is plotted vs. fo; in (B) 
the mean external input /o is fixed and v-i is plotted vs. e""'. The markers 
indicate m-l for FFNs of different sizes [m and t are given by the legend in (A)] 
obtained by numerical simulations of propagating synchrony. The dashed lines 
shows the approximation of m-l given by Equation (46) (which is independent 
of to and e); the solid lines indicate v-i = p*G2ei values of pj^ and G2 are found 
semi-analytically, by numerically identifying the bifurcation point of the 
analytically derived iterated map (Equation 13) for the different network setups 



(both analytical estimates are corrected for the influence of inputs from the 
preceding layer up to the first order). Panel (C) shows the fraction p'™ of 
neurons in a layer that participate in the propagation of a synchronous signal 
vs. a [(Equation 10); main panel] and vs. the spontaneous firing rate v (inset). 
Data from different network setups are plotted without distinction as black 
dots in the main panel and with distinction by different colors and symbols in 
the inset (see legend): Simulations are repeated for different layouts of the 
external network (fo e {1, 3, . . . , 11) mV; e""' e {0.1 , 0.125, 1 .0) mV). The 
solid lines indicate Pf ((i^) = fp (a) as given by Equation (53). The layer size m 
as well as the coupling strength e influence ^"^'^ only weakly, p'^^'^ depends on 
the network setup mainly through a or, equivalently, through v (cf. Equation 9): 
Measurement values from different network setups largely collapse to the 
graph of the function pf (m-l) = fp (a). For further explanations see text 
(section 3.1.4). 
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such that 



+ Erf 



M-i - 0 + 



= i[Erf(a) + Erf(l^-a)] 



= - [Erf (a) + Erf (/-^(a) - a)] =: fp{a). 



(51) 
(52) 
(53) 



In Figure 5C we compare the above predictions with direct 
numerical simulations: For different layer sizes m, coupling 
strengths € and layouts of the external networks (i.e., different 
values of Iq and €™*), we detect whether propagation of a syn- 
chronous pulse is possible and if so, we numerically determine 
the average fraction of participating neurons as well as the spon- 
taneous firing frequency. We find that indeed the size of the 
synchronous pulse is determined essentially by the quotient a = 
and Equation (53) is a reasonable estimate of the average 
fraction of neurons spiking in each layer. With increasing a the 
fraction of participating neurons increases, it thus decreases with 
spontaneous firing rate v see Figure 5C. For FFNs with low spon- 
taneous spiking frequency almost all neurons of a layer participate 
in the propagation of a synchronous pulse. 

3.2. FFNs WITH NON-LINEAR DENDRITES 

In this section, we investigate propagation of synchrony mediated 
by dendritic non-linearities. Although the mechanism underly- 
ing the propagation is generally related to that in linear networks, 
the discontinuities introduced by non-additive dendritic interac- 
tions prevent a similar analytical approach. In the first part of 
this section, we thus derive analytical estimates for the critical 
connectivity pfj^ in non-linearly coupled networks based on a 
self-consistency approach (see also Jahnke et al., 2012). In the sec- 
ond part, we study the transition from propagation of synchrony 
mediated by linear dendrites to propagation of synchrony medi- 
ated by non-additive dendritic interactions upon increasing the 
degree of non-linearity in the networks. In the last part, we eval- 
uate the robustness of the analytical estimates with respect to the 
layout of the external network. 

3.2. 1. Analytical derivation of critical connectivity 

Neurons with non-additive dendritic interaction process exci- 
tatory input by a non-linear dendritic modulation function 
0]VL (see section 2.1), i.e., synchronous inputs that exceed the 
dendritic threshold ©(, are amplified to an effective input of 
size K (cf Equation 4). Therefore the spiking probability of 
a single neuron due to a synchronous input of strength x, 
Pf {ois!l{x)), is discontinuous and an approach based on expan- 
sion of p/(-) is inappropriate. To derive an analytical expres- 
sion for the critical connectivity in FFNs incorporating 
dendritic non-linearities, we consider the (average) fraction of 
neurons of one layer, py, that receive an input x larger than 
the dendritic threshold, x > &h, due to the propagating syn- 
chronous pulse. If there is a stable (stationary) propagation of 
synchrony established, py is constant throughout the layers, which 
allows us to formulate a self-consistency equation. The basic 
derivations have been published recently (Jahnke et al, 2012) 



and wiU be briefly reviewed in the following for the readers 
convenience. 

For sufficiently small dendritic thresholds 0;, and sufficiently 
large k, the spiking probability of a neuron due to a sub-threshold 
input is small compared to the spiking probability of a supra- 
threshold input. Therefore, we approximate the spiking proba- 
bility of a single neuron in response to a synchronous input of 
strength x by 



pf (aML(x)) 



P/(k) 
0 



if x>@b 
otherwise 



(54) 



i.e., we assume that somatic spikes due to the synchronous pulse 
are exclusively generated by dendritically enhanced inputs. We 
denote the fraction of neurons that receive a dendritic spike by 
py. This may be considered as constant throughout the differ- 
ent layers if stable propagation of synchrony is enabled. Then the 
probability that a neuron receives exactly k inputs from the pre- 
ceding layer follows a binomial distribution B (co, pypf (k) p), 
where pypf (ic)p is the probability that (1) a neuron of the pre- 
ceding layer receives a supra-threshold input (py), (2) a somatic 
spike is elicited by that input (p>f (k)) and there is a connection 
from this spiking neuron to the considered neuron of the follow- 
ing layer (p). So we can formulate the self-consistency equation 
for py. 



Py 



k= [©jAI 



(PyP/(K)p)'^(l -PyP/(K)p)" ^ (55) 



To solve Equation (55) we approximate the binomial distribu- 
tion by a Gaussian distribution with mean 8 := dopyppf (k) and 
standard deviation 05 := ^8(1 — pyppf (k)), which yields 



Py 



1 + 



where we defined 



0fc/€ 



Oh 



UipyPPf (k) - 9bA 

^mpyppf (k) (1 - pyppf (k)) 



(56) 

(57) 
(58) 



as the difference between the average number of inputs (8) and the 
number of inputs needed to reach the dendritic threshold (0fc/e) 
normalized by the standard deviation of the number of inputs 
(as). Solving definition (Equation 58) forp and replacing py by 
Equation (56) yields 



PNL 



n^6 -I- 2@b + njn^t^ + 4@b 



p^(K)€(«2 + M)(l + Erf(^)) 



(59) 



which is the connectivity pjvi where stable propagation of syn- 
chrony with some given n (or, equivalently, some given py; 
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cf. Equation 56) is established. We note that a propagation of 
synchrony mediated by dendritic spikes requires 



(60) 



(otherwise even the input caused by a synchronized spiking of 
all neurons of a layer in a fully connected FFN (p = 1) is not 
sufficient to reach the dendritic threshold ©;,). 

For parameters fulfilling the inequality (Equation 60), pNiin) 
has a global minimum (see Appendix) and the critical connec- 
tivity p*j^^, again defined as the smallest connectivity that allows 
for a stable propagation of synchrony, matches that global mini- 
mum: any connectivity pNL above the minimal connectivity p]^^ 
has two preimages mi and ni corresponding to the both non- 
trivial fixed points Gi and G2 of the iterated map for the average 
group size (cf Figure 1 and section 2.4). However, there exists 
smaller connectivities for which a stationary propagation can be 
established. At the global minima p*^^ both preimages «i and 
M2 collapse to «* = mi = M2 and correspond to the fixed point 
G = Gi = G2 of the iterated map at the bifurcation point of the 
tangent bifurcation. Here the transition from the regime where 
no propagation of synchrony is possible to the regime where a 
propagation of synchrony is enabled takes place. For pjsn smaller 
than there are no preimages (i.e., a stationary propagation of 
synchrony mediated by non-additive dendritic interactions can- 
not be established); this scenario correspond to the absence of 
the non-trivial fixed points of the iterated map for connectivities 
below the tangent bifurcation. 

In the following we wiU obtain the minima of pM, (i-e-> the 
critical connectivity p^^) in the limit of large layer sizes oj and 
small coupling strength €. We first derive an approximation of 
Equation (59) (cf Equation 62), determine the validity range of 
this approximation (cf Equation 69) and finally obtain an esti- 
mate for the critical connectivity (cf Equation 71). As before, we 



fix the maximal input eco to each neuron to preserve the network 
state and expand Equation (59) in a power series around 6^0 
and u> ^ 00. Considering the leading terms yields 



PNL ~ PNI. a '■ = 



26,, 



1 -I- M, 



P/(K)6M i+Erf(-^) 



(61) 



Further a propagation mediated by dendritic spikes (as intro- 
duced above) requires that the layer size co and the coupling 
strength € are sufficiently large such that a sufficiently large frac- 
tion of neurons of each layer receive a total input larger than 
the dendritic threshold 0^. In particular for diluted FFNs, this 
requirement translates to eco ^ and Equation (61) simplifies 
further to 



pNL, b '■ = 



2©fe 



1 + n 



p^(ic)€«i^E^f^^^ 



(62) 



Whereas pivi has always a global minimum for €M > ©b, this does 
not hold for the approximation p^i, fc, e.g., (cf also Figure 6C) 



Km (pNL.fc) 



(63) 



However, we will now show that pjvi, b has a (local) minimum if 
(and only if) 6 e ^0, j which approximates the global min- 
imum of pjvi and therefore serve as an estimate for the critical 

= 0 yields 



connectivity. Starting with 




(64) 



e = 0.075 mV 
e = 0.3 mV 
e = 2.0 mV 
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FIGURE 6 I Determining tlie critical connectivity in FFNs with 
non-additive dendritic interactions. (A) For a given setup, i.e., for a given 
dendritic threshold ©(, and coupling strength ce < the corresponding 
n* (or equivalently Py-, cf. Equation 56) is found by Equation (64). The solid 
line indicates n* vs. e (left vertical scale), the dashed line Py vs. e (right 
vertical scale) and the markers n*(<;) for e = {0.075, 0.3, 2.0) mV (see 
legend). IHere, the dendritic threshold is ©i, = 4mV, such that the estimate 
(Equation 64) is valid within the range e e (0, 2.55JmV; Equation (69)1 (B) 
Knowing n* allows to evaluate P(^) e j, 1 according to Equation (70). 



Panel (B) shows p (cf. Equation 70) vs. ce (solid line, lower horizontal axis) 
and p vs. n* (dashed line, upper horizontal axis), respectively. (C) Finally, 
the critical connectivity pj^^ is obtained by Equation (71) which depends on 

P^nel (C) shows the connectivity Pfjj^ldashed; Equation (59)1 and 
its approximation pr^^^ (, Isolid; Equation (62)1 vs. n; for t e (0, e™""], pr^^^ i, 
has a local minimum which agrees with the global minimum of Pnl- The 
markers indicate the critical connectivity pj^^ obtained by the procedure 
described in (A) and (B). For further explanations see text 
(section 3.2.1). 
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and «* specifies the extremum oipuL, !>(«)• The second derivative 
oipNL, b(n) at the extremum n* given by Equation (64) satisfies 



'^Pnl. b 



drfi 



2m*, 



p;(K)co(l + Erf[^]) 



> 0 



(65) 



if «* > 0 such that the extremum actually is a minimum. Taken 

together, for a given setup, i.e., for given dendritic threshold 0;, 
and coupling strength e, the transcendent Equation (64) defines 
n* which maximizes or minimize pNL,b(f) and if additionally 
n* > 0 the extremum pML. b («*) is a minimum. 

Differentiating the right hand side of Equation (64), 



dfi.n*) 
dn* 

d'fjn*) 



n* -e^J- I 1 + Erf 



= n* + (l + «*^) 



4~2. 



1+Erf 



(66) 
(67) 



shows that / («*) (as defined in Equation 64) is (1) minimal for 
«* = 0 and (2) monotonicaUy increasing for «* > 0; according to 
Equation (64) the minimum «* = 0 corresponds to 



20b 



0.640;,. 



(68) 



The left hand side of Equation (64), i.e., ^/©b/f , is monotonicaUy 
decreasing with € from infinity to zero. Thus Equation (64) has a 
solution for any 



(69) 



and p^j^ := p*^^ ^ («*) is the (local) minimum of Equation (62) 
and provides an estimate for the critical connectivity, the (global) 
minimum of Equation (59). 

For better readability we define the function P(-), 



2jt 



(70) 



where n* = rf {^j^ as given by Equation (64). We note that 

P can also be considered as a function of «*. By combining 
Equations (62), (64), and (70) we obtain the critical connectivity 

®b 1 



(71) 



The function P(-) itself is monotonicaUy decreasing with e in the 
validity range e e (0, e™^] of the above approximation: within 



this interval n* > 0 and -j4r/ («*) > 0 and thus the derivative 



dp 



dp 
dn* 



dn* 



d^/@bjl 



dyJWbfl 
dd 



»2 



df{n*) 
dn* 



< 0. 



Consequently P assumes its minimum 



pmin ^ P („* = 0) = 



(72) 

(73) 
(74) 

(75) 



for € : 



201, 



and increases monotonicaUy with decreasing 



e against its asymptotic value 



P""^= lim 

n*— >-oo 



('+'45])-"" 



= 1. (76) 



Thus the critical connectivity is bounded by 

®b _ @b 



Pf (k) c» 



■P*NL<2- 



Pf (k) 



l-p" 



(77) 



and converges to the lower bound p° for smaU t and to its upper 
bound 2p'^ for large e. 

In Figure 6 we visualize the determination of the critical con- 
nectivity (Equations 64, 70) and Equation (71). The critical 
connectivity obtained with the approach presented above agrees 
weU with simulation data (cf. Figure 7). 

3.2.2. Transition from linear to non-linear propagation 

In the previous section we derived analytical estimates for the 
critical connectivity p'^^ in FFNs with non-additive dendritic 
interactions; is determined by (1) the setup of the FEN (i.e., 
the layer size w and coupling strength t; c£ Figure?), (2) the 
parameters of the non-linear modulation function (i.e., the den- 
dritic threshold 0;, and enhancement level k) and (3) the layout 
of the external network (i.e., the mean external input Iq and its 
variance, which is proportional to e^'^). In this section, we dis- 
cuss the influence of the parameters of the non-linear modulation 
function and study the transition from a regime where propaga- 
tion of synchrony is mediated by dendritically enhanced inputs to 
a regime where the majority of inputs is processed linearly. 

In general, with increasing threshold ©b more and more inputs 
are needed to reach this threshold and consequently the critical 
connectivity p^j^ increases. If ©;, exceeds |Xl, which is the average 
input to the neurons if a synchronous pulse propagates in lin- 
early coupled FFNs (cf Equation 45 and Figure 5), propagation 
mediated by linearly processed spikes is enabled for lower connec- 
tivities than propagation mediated by dendritic non-linearities. In 
this regime the linearly summed inputs (forp = p|) are sufficient 
to maintain propagation of synchrony, but are not sufficient to 
cross the dendritic threshold. Increasing ©^ even further has no 
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FIGURE 7 I Critical connectivity in FFNs with non-linear dendrites. The 

panels show (A) the critical connectivity p^j^ vs. the layer size (u for 
different coupling strengths (c = {0.05 mV (red), 0.1 mV (cyan), 0.125 mV 
(green), 0.2 mV (blue), and 0.4 mV (black))) and (B) pj^^ vs. the coupling 
strength e for different layer sizes (m = {50 (red), 100 (cyan), 150 (green), 
200 (blue), and 400 (black))). The points indicate the minimal connectivity 
for which a synchronous pulse propagates from the first to the last layer in 
an FFN with m = 20 layers in at least 50% of n = 30 trials. The critical 
connectivity given by Equation (71) (solid lines) is in good agreement with 
the computer simulations. (C) The critical connectivity is confined to the 
interval p^^ e [p°, 2p°] [indicated by the gray area for m = 150 (green), cf. 
Equation (77)1 and approaches its lower bound for small e and its upper 
bound for large e. Like in lineariy coupled networks the critical connectivity 
decays inversely proportional to layer size, p^^ oc m"' , (cf. also Figure 3), 
but the scaling with coupling strength is more complicated, 
p^^ ■ 1/p(^): the factor p^^) e |0.5, 1] [cf. Equation (70) and 
Figure 6] measures the deviation from the algebraic decay (as found in 
linearly coupled networks). In this figure the parameters of the external 
network are fixed to fo = 5 mV, v""' = 3 kHz, ce'"<* = 0.5 mV. 



influence on the critical connectivity p^j^, here a propagation of 
synchrony is possible forp > discussed in section 3.1. 

We illustrate this transition from non-linear to linear prop- 
agation in Figure 8A: We start with large &b = [^l such that 
propagation is enabled for p ~ ^l^o set k = [ii. In fact, 

the linear critical connectivity pj slightly under-estimates the 
observed critical connectivity pf^^ as it does not account for 
the saturation of the non-linear modulation function, i.e., for 
the cutoff GMiix) = k; of inputs x > k. With decreasing the 
critical connectivity is substantially reduced and well approx- 
imated by Equation (71). Propagation of synchrony is now 
mainly mediated by dendritically enhanced inputs as described 
in section 3.2.1. The inset illustrates the impact of decreas- 
ing the dendritic threshold on the iterated map. Initially, 
for ©ft = |Xi = K, the iterated map for linearly coupled and 
non-linearly coupled FFNs is similar; with decreasing ©(, the 
jump like rise in the iterated map is shifted to lower group 
sizes and consequently the bifurcation point is shifted to lower 
connectivities. 

The non-linear modulation function ctnl(-) (cf Equation 4) 
saturates for strong inputs, thus the enhancement level k defines 
the maximal (effective) input to a neuron and pf (k) is an upper 
bound for the spiking probability of any neuron in response to 
incoming inputs. This implies that in contrast to linearly cou- 
pled FFNs, the average size of a propagating synchronous pulse, 
Ym,) given by the product of the probability of a neuron receiv- 
ing sufficiently strong input to reach the dendritic threshold (py; 



cf Equation 56), the spiking probability due to that input [pf (k)] 
and the layer size to, is bounded from above by 

YNL = Pypf (K) M < MP/ (K) =: Y™''. (78) 

This bound decrease with decreasing k as illustrated by Figure 8B 
(inset), where we compare the iterated maps for different values of 
K. pf (k) also influences the critical connectivity p^^ (cf. Equation 
71): For small k the spiking probability py (k) is low and thus 
p*^^ is large (it may even exceed pp. With increasing k also P/(k) 
increases and consequently the critical connectivity p^^ decreases; 
for very large k the spiking probability pf (k) approaches 1 (cf 
Equation 14) andp^^ saturates (cf Figure 8B). 

In Figure 8C we show the critical connectivity for an additive 
enhancement by a constant A, i.e., inputs exceeding the dendritic 
threshold ©^ are increased by the constant value A = k — ©;,. 
For small v. the critical connectivity p^^ is relatively large and 
may exceed p^ due to the low saturation level of the non-linear 
modulation function 0m,(-) (cf also Figure 8B). As mentioned 
above, with increasing k, also P/(k;) increases and the critical 
connectivity p^^ decreases. However, for large k and thus large 
dendritic threshold ©(, propagation of synchrony mediated by 
linearly processed spikes is possible for lower connectivities than 
propagation mediated by dendritic non-linearities. Consequently, 
p*^^ converges toward p^ (cf also Figure 8A). 

3.2.3. Influence of external network 

In section 3.2.1 we derived an estimate of the critical connectivity 
p^^ for FFNs with non-additive dendritic interactions. So far we 
discussed the influence of the setup of the FFN (layer size m and 
coupling strength e) as well as the parameters of the non-linear 
modulation function Ojvi (dendritic threshold ©;, and enhance- 
ment level k). In the current section, we focus on the remaining 
determining factor, the layout of the external network. How does 
the critical connectivity change with the mean external input Iq 
and external coupling strength 6™* and how well are these changes 
covered by our analytics? 

For the derivation of p^^ we assumed that somatic spikes are 
elicited exclusively by dendritically enhanced inputs (cf Equation 
54) and thus the critical connectivity depends on the layout of 
the external network only via pf (k) (cf also Equation 71), i.e., 
on the average spiking probability of a neuron receiving an input 
larger than the dendritic threshold x > Qy. For sufficiently small 
pf (k), p^^ > 1 and propagation of synchrony is not possible. 
With increasing pf (k) the critical connectivity decreases and for 
pf (k) — > 1 it converges to ©(, (emP [0i,/6])^\ independent of 
the external network. 

In the regime of low spiking rates, changing the mean exter- 
nal input 7o simply shifts the distribution of membrane poten- 
tials Pv{V) (which is a Gaussian distribution centered at Iq; 
cf Equation 8). Thus, with increasing 7o, pf (k) increases and the 
critical connectivity p^^^ decreases. 

In Figure 9A we show the critical connectivity for different 
e'^^ [which determines the width of Pv{V)] vs. the mean exter- 
nal input Iq. For Iq = 0 — k (such that the sum of a dendritically 
enhanced input and the center of the distribution of membrane 
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FIGURE 8 I Transition from linear to non-linear propagation. The figure 
shows the critical connectivity pj^,^ vs. the parameters of the non-linear 
modulation function a^L (cf. Equation 4) for different network setups (color 
code, see (C)). The lines are the theoretical predictions for p^j^ [solid, 
Equation (71)1 and p* [dashed, Equation (32)1. The markers indicate the 
minimal connectivity for which a synchronous pulse propagates from the first 
to the last layer in an FFN (/q = 5 mV, v"''' = 3 kHz, e""' = 0.5 mV) with 
m = 20 layers in at least 50% of n = 30 trials. The insets illustrate the effect 
of changing ©i, and k on the iterated map, cf. Equation (13), where 
connectivity is kept constant. (A) Critical connectivity vs. dendritic threshold 
0b for constant enhancement level k = \Li^ 13.7 mV (cf. Equation 50). If the 
dendritic threshold ©j, is sufficiently small such that Pf (©b) <K Pf (k) 
(cf Equation 54), the propagation of synchrony is mainly mediated by 
non-linear enhanced inputs and the critical connectivity can be estimated by 



Equation (71). For large ©b the probability that an input from the preceding 
layer exceeds the dendritic threshold is very low, propagation of synchrony is 
mainly mediated by linearly processed inputs and the critical connectivity is 
given by Equation (32). Between these scenarios (for moderate ©b) there is a 
"transition regime," where linear and non-linear propagation mix [similarly in 
(C)l. (B) Critical connectivity vs. enhancement level k for constant threshold 
©b = 4 mV. For small enhancement levels k the (maximal) spiking probability 
of a single neuron, pf (ic), is small and thus the critical connectivity p^,^ is 
large. With increasing k, Pf (k) increases and thus pj^^ decreases; for large k, 
Pf (k) -> 1 (a neuron will almost surely spike upon the receipt of a 
non-linearly enhanced pre-synaptic input) and the critical connectivity 
saturates. (C) Critical connectivity vs. enhancement level k for an additive 
enhancement by a constant A = k— ©(, = 4 mV. For further explanations see 
text (section 3.2.2). 




lo(mV) 



(mV) 



FIGURE 9 I Dependence of the critical connectivity p'^i^ on thie layout of 
the external networl<. (A,B) The lines indicate the theoretical prediction for 
p^j^ given by Equation (71) and agree well with the data from direct 
numerical simulations (markers; FFN with m = 150, e = 0.2 mV, ©b = 4 mV, 
K = 11 mV, m = 20). Panel (A) shows the critical connectivity vs. the mean 
external input for fixed t""^ and panel (B) shows the critical connectivity 
vs. for fixed mean external input fo. The gray line indicates the minimal 
critical connectivity obtained for pf (k) = 1 . With increasing mean (external) 
input lo the distribution of membrane potentials PyiV) is shifted toward the 
somatic threshold ©, thus the spiking probability pf (k) upon the reception 
of a non-linear enhanced input increases and the critical connectivity p^^^ 
decreases. For /q = © — k, pf (k) a; 0.5 (cf Equation 80) and pj^^ is largely 
independent of the layout of the external network [blue solid line in (B); cf. 
also (A) where all curves coincide]. Further explanations see text 
(section 3.2.3). 



potentials equals the somatic threshold @),pf (k) simplifies to 



PfiK) = - I Erf 



"©-/o" 


+ Erf 


cr 





K- e + io' 

a 



(79) 



lErff^ 
2 \ a 



(80) 



and thus in the regime of low spiking rates, i.e., (8 — Iq) /a JS> 
1, pf (k) ~ 0.5 independent of the width of the distribution of 
membrane potentials. Consequently, all curves for different 
coincide at this point. For Iq > & — k the majority of neurons 
(>50%) would spike upon receipt of a dendriticaUy enhanced 
input. Thus pf (k) increases and therewith the critical connectiv- 
ity decreases upon decreasing t'^^ . In the limit of € ^ 0, Pv(V) 
converges toward a 8-distribution centered at /q and pf becomes 
a step-function 



Pf (k) 



0 K <@-Io 

1 K > © - 7o 



(81) 



such that the critical connectivity is either constant and minimal 
for 7o > 0 — or it diverges (no propagation possible) for Iq < 
& — K (cf Figure 9A; magenta curve). 

In Figure 9B we illustrate the effect of changing €™' on the 
critical connectivity for constant Iq. As discussed above for Iq = 
& — K, pf (k) and thus p'^^ are rather independent of 6™' and 
for Iq > @ — K the critical connectivity increases with €™*. For 
lo < © — K an increase of the width of the distribution of mem- 
brane potentials shifts the membrane potential of more and more 
neurons toward the relevant interval [8 — k, ®] and thus pf (k) 
increases and the critical connectivity decreases. 

For the derivation of we have assumed that the ground 
state dynamics is essentially not influenced by the spontaneous 
activity of the FFN itself (i.e., [i = Iq and a = €''''V2t™v™' ). As 
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discussed in section 3. 1 .3, we can correct the results for such influ- 
ences. However, since in non-linearly coupled FFNs the impact of 
(non-linearly enhanced) synchronous activity is much stronger 
than the impact of spontaneous activity (which is irregular and 
not amplified by non-additive dendritic interactions), we find 
that the deviations between the corrected and uncorrected version 
ofp^£ is negligible. 

Finally, we compare the critical connectivity for networks with 
and without non-additive dendritic interactions: The factor 



P*NL 



(82) 



measures how much the connectivity within the FFN can be 
reduced by introducing non-additive dendritic interactions. It is 
independent of the layer size m and becomes maximal in the limit 
of small coupling strengths 6 as P (©fc/e) — > P™™ = 1 for € — 0 
(cf. Equation 76). It increases with decreasing &h and increas- 
ing K (see discussion in section 3.2.2). In Figure 10 we show 
the influence of the external network. As discussed above, for 
small 7o, propagation of synchrony is not possible (the non-linear 
enhanced input is insufficient to elicit sufficiently many spikes in 
the layers of the FFN; white areas in Figure 10). With increasing 
7o, pf^i decreases and c"' increases. 

3.3. GENERALIZATIONS 

In the final section we discuss generalizations of the methods and 
results we derived. Compared to biological neurons, our models 
have simplifications which enable the analytical treatment, but 
might be suspected to be influential on the final result. These 
simplifications are the homogeneous delay distribution, the sim- 
plified initiation and impact of dendritic spikes, the limit of short 
synaptic currents and the sub-threshold leaky integrate-and-fire 




0.5 e™(mV) 



0.3 0.5 e™(mV) 



FIGURE 10 I Critical connectivity and reduction factor Panel (A) sliows 
the critical connectivity obtained from simulations of an FFN (o) = 150, 
e = 0.2 mV, m = 20) incorporating non-additive dendritic interactions 
Ob = 4mV, K = 11 mV; see also Figure 9). Within the white area, 
propagation of synchrony is impossible because even for a fully coupled 
chain the input to the next layer (limited by the saturation of the non-linear 
modulation function and the layer size) is insufficient. Panel (B) shows the 
reduction factor <f^^ (cf. Equation 82), the quotient between the critical 
connectivity in FFNs without and with non-additive dendritic interactions. 
The lines enclose the area for which the spontaneous firing is between 
V e [0.5, 1 .51 Hz obtained from simulations (solid) and low-rate 
approximation (cf. Equation 9; dashed). 



dynamics. Here, we verify that our results generalize to biolog- 
ically more detailed neurons without these simplifications. In 
particular, we show that the estimates for the critical connectivity 
hold. Further, we consider a qualitatively different dendritic inter- 
action function which assumes that the saturation is incomplete, 
i.e., beyond a region of saturation the impact of larger inputs 
increases. We show that the tools developed in the article are 
stiU applicable and reveal a new phenomenon, the coexistence of 
linear and non-linear propagation of synchrony. 

In the first part (section 3.3.1), we discuss the influence of 
inhomogeneous delay distribution and finite dendritic integra- 
tion windows. In the second part (section 3.3.2), we consider 
the non-linear modulation function with incomplete saturation. 
Finally, we consider biologically more detailed neuron models 
(section 3.3.3). 

3.3.1. Heterogeneous delays 

So far we considered FFNs with homogeneous delay distribution 
and dendritic modulation functions with integration window of 
zero length, i.e., only exactly synchronized inputs were possi- 
bly non-linearly amplified. Are these assumptions crucial for the 
obtained results? How does the critical connectivity change in the 
presence of heterogeneous delay distributions? 

To answer this question, we consider synaptic delays Xki (speci- 
fying the synaptic delay between neuron I and k) uniformly drawn 
from 



AT AT 

, t-l- 

2 2 



(83) 



where x is the mean delay. A direct consequence of heteroge- 
neous delay distribution is that the spikes of the propagating 
synchronous signal are not simultaneous (i.e., exactly synchro- 
nized) anymore. To describe the system accurately one has to 
consider additionally to the size (gi) also the temporal jitter (s,) 
of the synchronous pulse in the ith layer and investigate the two- 
dimensional iterated map for (gi, s;) (e.g., Diesmann et al, 1999; 
Gewaltig et al, 2001; Goedeke and Diesmann, 2008). However, 
even if the synchronous pulse is blurred out to a pulse packet with 
finite width, for sufficiently large connectivity stable propagation 
StiU can be obtained (see e.g., Gewaltig et al., 2001). 

For linearly coupled FFNs, with increasing width of the delay 
distribution, AT, the propagating pulse becomes broader and 
thus the critical connectivity increases (cf Figures 11 A,B; 
squares). However, the scaling with layer size (cf Figure 1 lA) and 
coupling strength (data not shown) is the same. 

Under the assumption that the width of the pulse packet stays 
bounded, one can derive a lower bound for the critical connectiv- 
ity. We assume that a pulse in layer i is perfectly synchronized and 
calculate the effective peak of the depolarization in the (i -|- l)th 
layer. Replacing the coupling strength 6 by the effective depolar- 
ization €' (derived below, cf Equation 89) in the estimate of the 
critical connectivity (cf Equation 32) one gains an estimate of 
the critical connectivity for systems with heterogeneous delays 
[Equation (90); shown in Figure 11]. Consider a perfectly syn- 
chronized pulse in layer Due to inhomogeneities in the delay, 
the inputs arriving at the {i + l)th layer are distributed uniformly 
in an interval of size AT (Equation 83). We assume that all inputs 
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FIGURE 11 I Robustness against heterogeneities in the response 
delays. (A) Critical connectivity vs. layer size for FFNs (m = 20, 
6 = 0.25 mV, /o = 5 mV, c""' = 0.5 mV) with additive (squares) and 
non-additive (85 = 4mV, k = 11 mV, At= 2.5 ms; circles) dendritic 
interactions. Different colors indicate different widths of the delay 
distribution (cf Equation 84). The solid lines indicate the critical connectivity 
pj^ corrected for inhomogeneous delay distribution (cf. Equation 90), the 
dashed line p^n^ for A7 = Cms. (B) Critical connectivity vs. width of delay 
distribution AT. Different colors indicate different setups of the FFN (red: 
o) = 275, e = 0.4 mV; green; m = 125, e = 0.25 mV; blue; ui = 200, 
6 = 0.1 mV). Solid and dashed lines are p' and pjj^ as before. 



arriving at a neuron of layer i + 1 are equidistantly distributed 
over [—AT/2, AT/2], i.e., the arrival time of the Zth of a total 
number of k inputs is 



AT AT 

~Y k - 1 



(/-I). 



(84) 



We consider the subthreshold dynamics only. Each single input 
depolarizes the neuron by an amount € and afterwards the mem- 
brane potential V{t) decays exponentially toward its asymptotic 
value (7o) with the membrane time constant t™ (cf Equations 
1, 2) until the next input arrives after a time interval 
(cf Equation 84). Thus the total (effective) depolarization caused 
by the sum of these k inputs at the end of the considered time 
interval 



(t+^):s 

A€;t = ^6 exp - 



1 AT 



;=i 

exp(-^,^) 



k - 1 
- 1 



(/ - 1) 



exp 



\ t" ly 
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1 



We consider the effective depolarization per input, in the limit 
of a large number of inputs k (k ^ 00), 



\ k 



AT 
=: C{AT)e 



1 — exp 



AT 



(87) 

(88) 
(89) 



Thus the correction factor C(AT) < 1 defined in Equation (89) 
relates the coupling strength f to the effective coupling strength e' 
in the presence of inhomogeneous delays. The critical connectiv- 
ity is then given by (cf Equation 32) 



Pl 



1 



1 



C(AT) X*6M 



(90) 



and this estimate agrees well with direct numerical simulations 
(cf Figure 11). 

For FFNs with dendritic non-linearities and inhomogeneous 
delays xj^i, one has to consider a finite dendritic integration win- 
dow Af''. Instead of amplifying only simultaneously received 
spikes (cf Equation 5), the sum of spikes within the time inter- 
val At is considered. We denote the sum of inputs to a neuron 
within the time interval [f — Af, t] by 



(91) 



where 



XAix) 



1 if X e A 
0 iix^A 



(92) 



is the indicator function and is the mth firing time of neuron 
I as before. If S^'(t) exceeds the dendritic threshold ©(, for some 
t = to, neuron k is depolarized additionally (to the depolarization 
arising from linear spike summation) by 



4'' (fo) 



■ (fo) 



(93) 



such that the total (effective) depolarization caused by an input 
x> @h equals k, modeling the effect of a dendritic spike; cf also 
section 3.3.3. After such an additional depolarization the den- 
drite becomes refractory for a time f'^^-'^^ and does not transfer 
additional spikes within the interval [fo, fo + f"^^'''^]. For Af = 0 
we recover the non-linear modulation function ctniCO given by 
Equation (4). Due to the finite dendritic interaction window, a 
delay distribution with AT < At affects the critical connectivity 
only weakly (cf Figure IIB). For AT > Af, some of the inputs 
received from the preceding layer upon a propagation of syn- 
chrony fall out of the dendritic interaction window A T and thus 
the critical connectivity increases. However, the scaling with layer 
size M (cf Figure IIB) and coupling strength 6 (data not shown) 
is practically identical with the scenario AT = 0. 

Before we discuss propagation of synchrony in biologically 
more plausible neuron models in section 3.3.3, we consider gener- 
alization of the non-linear modulation function in the following 
section. 

3.3.2. Coexistence of linear and non-linear propagation 

In this article, we employed a non-linear modulation func- 
tion aM,(f) that is linear for dendritic stimulation smaller 
than the dendritic threshold, 6 < &h, and constant (i.e., satu- 
rates) for supra-threshold stimulation, € > ©{, (cf Equation 4). 
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Biologically, if the linear inputs are transmitted despite the den- 
dritic sodium spike and are not shadowed by, e.g., an NMDA 
spike, they may lead to a second, later peak depolarization after 
the one generated by the sodium spike. Since our models replace 
depolarizations by jumps to the peak depolarization, we have to 
account for the later peak as soon as it exceeds the earlier one. 
In this part, we thus assume that if the synchronous input is so 
large that the depolarization it generates upon linear summation 
exceeds the depolarization k generated by the dendritic spike, this 
former is considered as the effect of the input. In other words, we 
assume that the dendritic modulation function continues linearly 
beyond k, i.e., we define 



€ for € < &I, 

K for 9fc < 6 < K 

€ for € > K 



(94) 



(c£ inset of Figure 12 A). 

The iterated map, mapping the number of active neurons in 
layer i to the average number of active neurons in layer i + 1 
may now have (depending on the system parameters) between 
one and five fixed points (cf Figure 12). As before. Go = 0 is a 
trivial fixed point corresponding to the level of absent activity 
and the only fixed point of the iterated map for small connec- 
tivity p. With increasing connectivity p, two additional pairs of 
fixed points Gi < G2 and G3 < G4 appear via tangent bifurca- 
tions. The first pair of fixed points, Gi and G2, correspond to 
the propagation of synchrony mediated by non-additive den- 
dritic interactions (as discussed in section 3.1), the second pair, 
G3 and G4, correspond to propagation of synchrony mediated 
by linearly processed inputs (as discussed in section 3.2). By 
further increasing the connectivity p, the fixed points G2 and 
G3 disappear via a tangent bifurcation (cf Figure 12A). Within 



the region, where five fixed points exists, both types of prop- 
agation of synchrony coexists (illustrated in Figures 12B-D): 
Synchronized pulses of size go < Gi typically decay to zero after 
a small number of layers. Pulse sizes with Gi < go < G3 typ- 
ically initiate propagation of synchrony with an average pulse 
size around G2 (where the propagation is mediated by non- 
additive dendritic interactions) and synchronous pulses of size 
go > G3 typically initiate propagation of synchrony with aver- 
age pulse sizes around G4 (linear propagation). For sufficiently 
large p, i.e., the fixed points G2 and G3 disappeared, a synchro- 
nized pulse of size go > Gi wUl initiate propagation of synchrony 
with pulse sizes around G4; in this parameter region the non- 
additive dendritic interactions essentially increase the basin of 
attraction of G4. 

Within the framework of our analytical tractable model, we 
neglect, e.g., the initiation time of a dendritic spike (in our 
model non-linear amplifications are instantaneous) or the differ- 
ent shapes of potential deflections caused by linearly and non- 
linearly processed inputs. Therefore, propagating synchronous 
signals mediated either by linear or non-linear dendrites differ 
only in their size. In biological more detailed models (briefly dis- 
cussed in section 3.3.3 below) both propagation types will be 
more distinct, e.g., the propagation frequency (speed) and the 
quality of synchrony of the propagating pulses are different (see 
also Jahnke et al., 2012). 

3.3.3. Biological more detailed models 

The model we mainly consider in this article has the advantage of 
being analytically tractable. Here we ask whether it over-simplifies 
the considered systems. More precisely, we study whether the 
results derived above, in particular the analytical estimates for 
the critical connectivity, generalize to biologically more detailed 
models. 
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FIGURE 12 I Coexistence of linear and non-linear propagation. 

(A) Bifurcation diagram obtained from Equation (13) for an FFN (m = 150, 
e = 0.225 mV) with a non-linear modulation function ctJ^^ with incomplete 
saturation Icf. Equation (94) and inset]. Panel (B) shows the iterated maps 
(Equation 13) for p = 0.5 with the different non-linear modulation functions 
considered in this article (linear coupling: green, dashed; non-linear coupling 
Ofji. red, dashed; modified non-linear coupling o'^j^: blue). Panel (C) depicts 



the development of the size of the synchronous pulse along the layers of the 
FFN (single trials). The blue and yellow regions are the basins of attraction of 
G2 and G4, respectively, derived from the data in panel (B). Panel (D) shows 
the probability p^°"'' of converging to the linear propagation regime (yellow 
area, blue line) and the non-linear propagation regime (blue area, red line) 
after m = 20 layers (p™™ is obtained from n = 150 runs with different 
networks and initial conditions). 
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The main assumption underlying our analysis of linearly cou- 
pled networks is a very general one, namely that synchronous 
single inputs sum up linearly: we assumed that the spiking prob- 
ability pf (•) of a neuron due to the reception of x synchronous 
inputs of size € equals the spiking probability due to the recep- 
tion of one single input of size y = xe. Therefore, the results will 
hold also for more complex neuron models, as long as the effect 
of a synchronous input pulse is approximately the sum of the 
effects of single inputs. In particular, if the spiking probability 
due to an input of strength x, pf{x), is sufficiently slowly chang- 
ing with X, according to Equation (24) the critical connectivity 
scales like (^m)^^ for sufficiently large layer sizes and small 
coupling strengths. To fully compute the critical connectivity, the 
actual form ofp/ (•) has to be known. Our leaky integrate-and-fire 
neuron with infinitesimaUy short current pulses approximates the 
behavior of a wide class of neuron models for which an analyti- 
cal derivation of p/(-) is impossible. StiU even for more detailed 
models, p/(-) is accessible for measurements in single neuron 
(computer) experiments. 

In Figure 13 we verify our predictions exemplary for two types 
of neuron models: We employ a model of conductance based 
leaky integrate-and-fire-type neurons with exponential input 
conductances (CB-type; see Appendix) and a Hodgkin-Huxley- 
type neuron model with alpha-function shaped input currents 
(HH-type; see Appendbc). The post-synaptic potential induced 



by single excitatory inputs is shown in panels (a) and (b) and the 
scaling of the critical connectivity p^ with eo) in panel (c): the 
scaling of p^ is well described byp^ oc (eco)^'. 

The main assumptions underlying our analysis of non-linearly 
coupled networks are (1) that the maximal spiking probability 
due to inputs which are subthreshold relative to the dendritic 
threshold, pf (0fc), is significantly smaller than the spiking prob- 
ability due to a suprathreshold input, pj (ic), and (2) that the 
temporal jitter of somatic spikes evoked by suprathreshold inputs 
is small such that synchronized inputs stay synchronized. Both 
conditions have been found to be satisfied in biological neu- 
rons (e.g., Ariav et al, 2003). Therefore, Equation (71) specifying 
the critical connectivity p^^ also holds for more detailed neuron 
models if these models incorporate biologically plausible features 
of fast dendritic spikes. To obtain a quantitative prediction of p^^, 
it is sufficient to estimate (a) the number of inputs needed to elicit 
a dendritic spike, ©fe/e, (b) the layer size m, and (c) the spiking 
probability due to the reception of a total input that is sufficiently 
strong to elicit a dendritic spike. 

To investigate the scaling of the critical connectivity p^^ in 
direct numerical simulations, we account for the effects of den- 
dritic spikes in the CB-type and HH-type: When the total exci- 
tatory input within the dendritic integration window exceeds 
the dendritic threshold level, a current pulse modeling the 
effect of a dendritic spike is initiated and causes an additional 




FIGURE 13 I Same scaling of propagating regime for networl<s of 
biologically more detailed neuron models. (A,B) Time course of tlie 
membrane potential of single neurons receiving inputs that are sufficiently 
strong to elicit a dendritic spike, with (non-linear model) and without (linear 
model) dendritic spike generation mechanism, for (A) a conductance based 
LIF-type neuron (henceforth: CB-type), and (B) a Hodgkin-Huxley-type neuron 
(HH-type). The insets show the observed peak of the induced postsynaptic 
potential (pEPSP) vs. the pEPSP expected from linear input summation 
(equivalent to the dendritic modulation function in the analytically tractable 
model). (C) Critical connectivity pl vs. cm in linearly coupled networks. For 
each value «o, we evaluated the critical connectivity for four different group 
sizes (u = 100, 300, 500, 700 and four different coupling strengths c = 0.3, 
0.6, 0.9, 1 .2 nS (CB-type; squares; lower horizontal axis) and « = 9, 18, 27, 
36 pA (HH-type; crosses; upper horizontal axis), respectively. The lines are 
fitted functions of the form (Icm)"^ . The analytical estimate given by 
Equation (24) holds in the limit of large layer sizes u> and small couplings c. 



therefore we exclude data points from the fitting where a single input yields 
an EPSP larger than 0.6 mV (CB-type: ce > 1 .4 nS; HH-type: e > 46 pA; these 
points are marked in gray). (D,E) Probability distribution of somatic spike 
times after stimulation of the neuron by an input which is sufficiently strong 
to generate a dendritic spike (D: CB-type, E: HH-type). We show exemplary 
two different configurations for the external inputs, which result in a total 
somatic spiking probability after dendritic spike generation of 0.97 (solid 
lines; set 1) and p' ^ 0.67 (dashed lines; set 2). equals the saturation level 
of the corresponding cumulative distribution function (shown in the insets). 
(F) Critical connectivity p^j^ vs. group size m (lower horizontal scale) and 
coupling strength c normalized by threshold (upper horizontal scale), 
respectively. The theoretical estimate of pjj^ (cf Equation 71) is a function of 
o), 9b/e and p' , therefore the predictions agree for both models and the data 
from direct numerical simulations are consistent with the theoretical 
predictions. [All simulations of FFNs in this figure are obtained for 
inhomogeneous delay distribution with AT" = 1 ms (cf Equation 83)]. 
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depolarization of the soma of the post-synaptic neuron (see 
Appendix for details; cf also section 3.3.1). In Figure 13 we com- 
pare the results of direct numerical simulations with the estimate 
given by Equation (71). The post-synaptic potential induced by 
single excitatory inputs is shown in panels (A) and (B). Panel (D) 
and (E) shows the spiking probability of a single neuron (in the 
ground state of the FFN), f/, due to an input exceeding the den- 
dritic threshold level; as examples we present two different setups 
with pf = {0.67, 0.97}. Panel (F) shows the scaling of p"^^ with 
layer size and coupling strength and the good agreement of the 
analytical estimate with direct numerical simulations. 

4. SUMMARY AND CONCLUSIONS 

Propagation of synchrony in feed-forward sub-structures that are 
embedded in randomly connected recurrent networks has been a 
research topic for more than two decades now [see, e.g., review on 
this topic (Kumar et al, 2010)] and it is hypothesized that such 
propagation possibly explain the emergence of spatio-temporal 
spike patterns and information transmission. 

In this article, we have analyzed diluted FFNs and investi- 
gated their capability to propagate synchrony. In addition to 
conventional additive (linear) input processing at single neurons, 
we considered non-additive dendritic interactions modeling the 
impact of fast dendritic spikes (Ariav et al., 2003; Gasparini et al., 
2004; Polsky et al, 2004; Gasparini and Magee, 2006). We emu- 
lated the influence of the embedding recurrent network which 
establishes the irregular ground state in the FFN, by random 
Poissonian inputs (van Vreeswijk and Sompohnsky, 1996, 1998; 
Brunei, 2000). This approach does not account for back- reactions 
of activity within the FFN on the embedding network. It is justi- 
fied as long as the connectivity and connection strength between 
the neurons of the FFN and the embedding network is low and 
weak compared to the feed-forward connectivity and connec- 
tion strength. The back-reaction then influences the activity of 
the embedding network only weakly and a robust propagation 
of synchrony can be achieved (Vogels and Abbott, 2005; Kumar 
et al., 2008; Jahnke et al, 2012). Yet, if the condition is not 
met, synchronous activity within the FFN may spread out over 
the embedding network and potentially cause pathological activ- 
ity ("synfire-explosions") (Mehring et al., 2003). For specifically 
structured networks also more complex interactions are possible, 
such as an enhancement of propagating synchrony (manuscript 
in preparation). 

In the main part of the article, we studied the propaga- 
tion of synchrony employing leaky integrate-and-fire neurons 
in the limit of temporally short synaptic inputs and homo- 
geneous synaptic delays. Synchronous pulses consist of exactly 
synchronized (simultaneous) spikes. This allows to investigate 
propagation of synchrony by considering the size of a syn- 
chronized pulse only, so that the analysis becomes analytically 
tractable. Nevertheless, in the second part of our article we also 
consider systems with heterogeneous coupling delays and tem- 
porally extended interactions. In agreement with the literature 
(e.g., Diesmann et al., 1999; Gewaltig et al, 2001; Goedeke and 
Diesmann, 2008), we observe that pulse packets tend to syn- 
chronize along the layers of the FFN so that the results of our 
simplified description are directly applicable. 



We derived scaling laws as well as quantitative estimates for 
the critical connectivity marking the bifurcation point between 
the regime where robust propagation of synchrony is possible 
and where it is not. In particular, based on a suitable series 
expansion we have shown that for linearly coupled FFNs the crit- 
ical connectivity decays inversely proportional to layer size and 
coupling strength. Moreover, the proportionality factor can be 
estimated from the ground state properties of the single neurons. 
The estimate agrees with direct numerical simulations within the 
biologically relevant parameter regime where (a) the spontaneous 
firing rate of the neurons is low and (b) the distribution of mem- 
brane potentials is broad (each neuron receives a huge number 
of almost random presynaptic inputs). If a synchronous pulse 
propagates along the layers of a linearly coupled FFN, most of the 
neurons of each layer participate in the propagation of synchrony, 
independent of the actual layer size, coupling strength or layout of 
the external network. 

For neurons incorporating non-additive dendritic interac- 
tions, the spiking probability as a function of the dendritic 
stimulation becomes discontinuous. Therefore, the analytical 
estimation of the critical connectivity in non-linearly coupled 
FFNs required a different approach than the treatment of lin- 
early coupled FFNs. We have shown that the critical connectivity 
decays inversely proportional to the layer size (as in linearly cou- 
pled FFNs), and we have derived the dependence on the coupling 
strength which is more complicated. The critical connectivity is 
completely determined by layer size, spiking probability of the 
single neuron upon the reception of a non-linearly enhanced 
presynaptic input and the number of inputs required to reach 
the dendritic threshold. Our results indicate that in presence of 
non-linear dendrites, neurons process synchronous inputs sim- 
ilar to threshold units. Such units have been previously used as 
simplified rate neuron models to study activity propagation in 
discrete time, e.g., in Nowotny and Huerta (2003); Leibold and 
Kempter (2006); Cayco-Gajic and Shea-Brown (2013). Because 
the non-linear modulation function saturates, FFNs with non- 
additive dendritic interactions allow for a sparser coding, i.e., 
only a sub-fraction of each layer (the actual size depends on 
the non-linear enhancement level) participates in the propaga- 
tion of synchrony. Whereas stable propagation of synchrony is 
possible in systems with and without dendritic non-linearities, it 
occurs in non-linearly coupled FFNs with substantially reduced 
feed-forward anatomy (reduced connectivity or reduced coupling 
strength) compared to linearly coupled FFNs. 

The analytic derivation of the critical connectivity is based 
on rather general assumptions: (a) the effect of a synchronous 
input pulse is approximately the sum of the effects of single inputs 
and (b) for networks with non-additive dendritic interactions the 
spiking probability due to non-linearly enhanced input is sub- 
stantially larger than due to a non-enhanced input. Therefore the 
predictions and estimates are directly applicable to networks of 
biologically more detailed neuron models. 

In our article we have shown that even highly diluted feed- 
forward structures are suitable to reliably support the directed 
and constrained propagation of synchronous activity. Such struc- 
tures occur naturally in sparse, random recurrent networks which 
are typical for the cortex. These structures might be enhanced 
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by simple synaptic plasticity to enable synchrony propagation. 
Fast dendritic spikes promote this propagation, as they selectively 
amplify synchronous inputs and are only weakly influenced by 
irregular background activity. 

Indeed, important candidate regions for the generation of 
propagating synchrony such as the hippocampus and other, neo- 
cortical regions exhibiting replay of activity (Nadasdy et al., 
1999; Lee and Wilson, 2002; Ji and Wilson, 2007; Xu et al., 
2011; Eagleman and Dragoi, 2012) are sparse and show synap- 
tic plasticity (Debanne et al., 1998; Kobayashi and Poo, 2004). 
Dendritic spikes as prominently found in, e.g., the hippocam- 
pus (Ariav et al., 2003; Gasparini et al, 2004; Polsky et al, 2004; 
Gasparini and Magee, 2006) trigger depolarizations and calcium 
influx sufficient to change synaptic strengths (Golding et al., 2002; 
Remy and Spruston, 2007) and the dendrites itself exhibit branch 
"strength potentiation," i.e., the strength of a dendritic spike on 
a dendritic branch exhibits experience- and activity-dependent 
plasticity (Losonczy et al, 2008; Makara et al, 2009; MiiUer et al., 
2012). 

Our work indicates that fast dendritic spikes reduce the 
required synaptic strength and connection density for replay of 
spike patterns. Moreover, their saturation and the resulting sparse 
coding might explain the observed variability during replay. Thus, 
in particular, our understanding of propagation along diluted 
feed-forward chains may now be combined with knowledge 
on synaptic plasticity and generation of activity accompanying 
replay (e.g., sharp wave/ripples) to gain an integrated mechanistic 
understanding for encoding, replay and memory transfer. 
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A. APPENDIX 

A.1 PROOF OF EXISTENCE OF A GLOBAL MINIMUM OF Pi^Lin) 

We will show that p^iin) as derived in Equation (59), 



pNL(n) 



p/(K)€(n2 + M) (l + Erf(^)) 
1 20f, + «26(^l+yi+ 



(A.1) 



(A.2) 



has a global minimum for tw > ©(,. In Equation (A.2) we defined 



4©b / ©f, 
a := 1 

6 V €M 



(A.3) 



For €00 > Oh, pNi is positive and continuous, and approaches 



lim (pNiin)) = oo. 



lim {pNL(n)) 



(A.4) 
(A.5) 



in the limit of large/small n. Further, the derivative of pML can be 
written as 



dn 

where we defined the functions 
/ 



pNL(n) = (2 - hi(n)) h2(n), 
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(A.9) 
(A. 10) 
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(A.12) 

(A.13) 
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For 60) > ©b, is smaller than two for sufficiently large n 

(cf Equation A.13) and thus the derivative of pMiin) becomes 
positive (cf Equation A.6). Consequently p^i approaches 
l/pf{K) from below for large n (cf also Equation A.5). This 
proves the existence of a global minimum of pNii^), because 
pNL > ^/pf (k) for sufficiently small n (cf Equation A.4). 

A.2 BIOLOGICAL MORE DETAILED NEURON MODELS 

In section 3.3.3 we consider biologically more detailed neu- 
ron models. In this appendix we present descriptions of these 
models including the parameters used for the numerical simu- 
lations in Figure 13. These simulations were done using NEST 
(Gewaltig and Diesmann, 2007), a simulator for spiking neu- 
ral network models (available at http://www.nest-initiative.org). 
We implemented new model classes within the NEST framework 
to handle conductance-based leaky integrate-and-fire neurons 
with double exponential input conductances as well as non- 
linear dendritic interactions (source code available from Sven 
Jahnke). 

A.2.1 CB-type model 

The CB-type model is a leaky integrate-and-fire neuron with con- 
ductance based synapses, augmented with a mechanism for the 
generation of current pulses mimicking the effect of a dendritic 
spike (see also Memmesheimer, 2010; Jahnke et al., 2012). The 
subthreshold dynamics of the membrane potential Vi of neuron I 
obeys the differential equation 



C! 



,dVi{t) 
dt 



+ gf(t) - y,(f)) + ij'Ht) + If. (A.15) 



Here, C™ is the membrane capacity, gj' is the resting conduc- 
tance, 1^'^^' is the resting membrane potential, E^'^ and are 
the reversal potentials, and gfit) and g^{t) are the conductances 
of excitatory and inhibitory synaptic populations, respectively. 
if^it) models the current pulses caused by dendritic spikes and 
if is a constant current gathering slow external and internal 
currents. The time course of single synaptic conductances con- 
tributing to gfit) and gf'it) is given by the difference between 
two exponential functions (e.g., Dayan and Abbott, 2001) with 
time constants x^' ' and t"*' ^ for the excitatory and t*^' ^ and t*^' ^ 
for the inhibitory conductances. Whenever the membrane poten- 
tial reaches the spike threshold ©;, the neuron sends a spike to its 
postsynaptic neurons, is reset to V;'^'*'^' and becomes refractory for 
a period f;"^*^^. Additionally to inputs from the preceding layer each 
neuron receives excitatory and inhibitory Poissonian input spike 
trains with rates v*^" and v'"; single inputs have coupling strength 
€™ and e'", respectively. 

To account for dendritic spike generation, we consider the 
sum gi /^f of excitatory input strengths (characterized by the cou- 
pling strengths), arriving at an excitatory neuron I within the time 
window Af for non-linear dendritic interactions. 



gl. At(f) 
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where X[t-At, f] is the characteristic function of the interval 
[t — At, t], is the kth firing time of neuron j and t denotes 
the synaptic delay. We denote the peak conductance (coupling 
strength) for a connection from neuron j to neuron I hy g^^. If 
gl, At exceeds a threshold gQ, a dendritic spike is initiated and the 
dendrite becomes refractory for a time window t°^'"^^. The effect 
of the dendritic spike is incorporated into the model by the cur- 
rent pulse that reaches the soma a time x'^^ thereafter. This cur- 
rent pulse is modeled as the sum of three exponential functions, 

lf^(t) = c(gAt) [-Ae"?^ +Be"?^ - Ce"?^] ,(A.17) 

with prefactors A > 0, 5 > 0, C > 0, decay time constants x^^' ^ , 
.j.DS,2^ .j.DS,3 jjjjj J dimensionless correction factor c (^Af)> where 
^Af is the summed excitatory input at the initiation time of the 
dendritic spike as given by Equation (A. 16). The factor c(^At) 
modulates the pulse strength, ensuring that the peak of the exci- 
tatory postsynaptic potential (pEPSP) reaches the experimentally 
observed region of saturation. At very high excitatory inputs, 
the conventionally generated depolarization exceeds the level of 
saturation, c {gAt) is zero and the pEPSP increases (cf. inset of 
Figure 13A). 

Parameters for Figure 13 

The single neuron parameters for the numerical sim- 
ulations are C]" = C™ = 400 pE = = 25 nS, 

yrest ^ yrest ^ .gj @^ ^ @ ^ _5q ^ref ^ ^ref = 3 ^s 

and V[^«^t = = -65mV. The reversal potentials are 

E^^ = OmV and = — 75mV and the time constants for the 
excitatory and inhibitory conductances are x"*'' = x^'^ = 2.5 ms 
and t"^'^ = x'^'^ = 0.5 ms. The parameters of the dendritic spike 
current are At = 2 ms, g'^ = 8.65 nS, x^^ = 2.7 ms, A = 55 nA, 
B = 64nA, C = 9nA, x"^'i= 0.2ms, t^'^-^ = 0.3 ms, 
^DS,3 _ 0.7 ms and t'^'^f-DS _ 5 2 ms and the dimensionless cor- 
rection factor is given by c(g) = max {l.5-^-0.053nS"^0}. 
For the first setup (/ 0.97) we set = /" = 250 pA, 
v^ = 2.4kHz, v'" = 0.6kHz, €^ = 0.6nS and €^" = 6.6nS; 
for the second setup (p^ ~ 0.67) we set if = 1° = 0 pA, 
v« = 20 kHz, v'" = 5 kHz, e""" = 0.6 nS and t'" = -6.6 nS. 

A.2.2HH-type model 

We employ a standard model provided by NEST 
("hh_psc_alpha"; Hodgkin-Huxley type neuron with alpha- 
function shaped postsynaptic currents) and incorporated a 
dendritic spike current as in the CB-Model. The membrane 
potential Vi of neuron I obeys the differential equation 

C'r^=lf\t)+If(t)+Il(t)+If 



dt 



+ If''it) + If'(t)+lfHt) 



(A.18) 



For clarity we drop the index I in the following; all quantities refer 
to some neuron /. In Equation (A.18), 



iHt) = g'^nit)* - V(t)] 
I^it) = / [E^ - Vit)] 



(A.20) 

(A.21) 



specify the Na+ current, the K+ current and leak current. The 
dynamics of the gating variables m, ti and h are governed by 



dmit) 

dt 
dh{t) 

dt 
dn{t) 

dt 



= ttmCO [1 - m{t)] - PmCOmCO 

= a^{t)[\-h{t)]-m)h{t) 
= a„(0[l-«(0] -Pn(On(0, 



where the voltage dependencies are given by 
0.01 [V(t) + 55] 



a„(0 = 



l-exp[-^] 



P„(t) = 0.125 • exp 



V(t) + 65 
80 



0.1 [V(t) + 40] 
l^expP^ 



Pm(0 = 4 • exp 
ah(t) = 0.07 • exp 
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(A.22) 
(A.23) 
(A.24) 

(A.25) 

(A.26) 
(A.27) 

(A.28) 
(A.29) 

(A.30) 



I^\t) 



g^^mit)^h(t) - Vit)] (A.19) 



In Equations (A.25-A.30) V(t) := is the value of membrane 
potential normalized by 1 mV. Spikes are detected by a com- 
bined threshold-and-local-maximum search, if there is a local 
maximum above a certain threshold of the membrane poten- 
tial, 17® = 0 mV, it is considered a spike (for more details see 
the NEST manual and the model implementation available at 
http://www.nest-initiative.org). After a synaptic delay time x a 
spike initiates an alpha-function shaped current pulse at the post- 
synaptic neurons. The total excitatory and inhibitory input to 
neuron I is given by 

= E'T^. [-^] ® - ^r] (A.31) 

k 

j'"« = E^r:^«p[-:^!;^]®[^-^r]- (A.32) 

k 

where €™ > 0 (e^" < O) is the strength of the fcth arriving excita- 
tory (inhibitory) spike at neuron /, tf' (tj^) denotes the reception 
time of that spike and e is the Euler constant [the currents I^(t) 
and r^(t) are normalized such that an input of strength € = 1 pA 
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causes a peak current of 1 pA] . The time constants and t'° 
are the synaptic time constants. As before, we account for den- 
dritic spike generation by considering the sum of excitatory input 
strengths received by neuron / within the time window At, 

«At(t) = J2^f^it-^t,t]{ti + -^)- (A.33) 
k 

If this sum exceeds the dendritic threshold 7®, a dendritic spike is 
initiated and we model its effect is by the current pulse 

starting after a delay time x^^ after the initiation time of the den- 
dritic spike. The correction factor c(€At) modulates the pulse 
strength such that the depolarization saturates for suprathresh- 



old inputs imtil the effects of linearly summed input exceed the 
effects of the dendritic spike (cf inset of Figure 13B). 

A.2.3 Parameters for Figure 13 

As before, we consider homogeneous neuronal properties. 
The single neuron parameters for the numerical simulations 

are C™ = 200 pF, = -77 mV, = -70 mV, E^^ = 50 mV, 
g^ = 3600nS, g^ = 30nS, ^'^^ = 12000 nS, x'=^ = 2ms and 
x'" = 2 ms. The parameters of the dendritic spike current 
are Af = 3.5ms, 7®= 270pA, t'^s = 2.7ms, A = 27.5nA, 
B = 32nA, C = 4.5nA, t^S'i = 0.2 ms, t^^'^ = 0.3ms, 
.jDS,3 _ 0.7 ms and f''^'^^ = 5.2 ms and the dimensionless cor- 
rection factor is given by c(€) = max {1.54 - € ■ 0.002 pA"\ O}. 
For the first setup (/ « 0.97) we set = 500 pA, = 3 kHz, 
v'" = 3 kHz, €™ = 20 pA and e'" = -20 pA; and for the sec- 
ond setup (/RiO.67) we set 7° = 250pA, v«^^=10kHz, 
v'° = 10 kHz, e^'' = 20 pA and €>" = -20 pA. 
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